Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
49 commits
Select commit Hold shift + click to select a range
4de52af
feat(expansion): add rolling single-world workload execution
yuecideng Nov 14, 2026
2cbe6a5
fix(expansion): own validated metadata inputs
yuecideng Nov 14, 2026
61f922d
fix(sensors): wait for contact sample before permission checks
yuecideng Nov 14, 2026
4723cba
feat(expansion): expose task ports and measured ghost comparison
yuecideng Nov 14, 2026
ab04d97
feat(graspkit): scope pose NMS memory budgets per task
yuecideng Nov 14, 2026
bc5ae3f
perf: reduce single-world planning and row execution overhead
yuecideng Nov 15, 2026
504f44f
docs(expansion): document native ghost preview and WebRTC follow-up
yuecideng Nov 15, 2026
bddf4d7
fix(physics): remove direct GPU API configuration override
yuecideng Nov 15, 2026
24c21fb
feat(expansion): preview confirmed results with native ghost rendering
yuecideng Nov 15, 2026
80d2d51
feat(expansion): collect and preview bounded parameter distributions …
yuecideng Nov 15, 2026
39201fe
fix(motion): preserve planning contexts and phase velocities
yuecideng Nov 15, 2026
85704e1
fix(expansion): preview recorded mesh scenes and native runtime metadata
yuecideng Nov 15, 2026
239a3d5
feat(expansion): compose native single-world workload collection
yuecideng Nov 15, 2026
78dc36c
fix(expansion): isolate workload grasp frames and reset settling
yuecideng Nov 15, 2026
7f97064
feat(expansion): expose declarative collection and preview entrypoints
yuecideng Nov 15, 2026
ed01657
Merge latest main and fix expansion CI regressions
yuecideng Nov 16, 2026
722e385
fix(sim): collect released native views before closing their scene
yuecideng Nov 16, 2026
f3e08ae
fix(expansion): address admission, acceptance and preview review find…
yuecideng Nov 16, 2026
8c244cd
fix(expansion): sync main and preserve placement phase paths
yuecideng Nov 16, 2026
2dfcbed
fix(expansion): stabilize hold timing and joint selection checks
yuecideng Nov 16, 2026
565d981
fix(gensim): clear supports and settle opening before release
yuecideng Nov 16, 2026
4a0f34c
merge: sync main before expansion performance fixes
yuecideng Nov 17, 2026
33fe553
test(gensim): align binding coverage with release clearance defaults
yuecideng Nov 17, 2026
bf4a254
fix(expansion): preserve selected rows and batch measured validation
yuecideng Nov 17, 2026
d083f61
merge: preserve concurrent GenSim binding coverage
yuecideng Nov 17, 2026
c3e6cd1
test(expansion): use native monitors in hold sampling coverage
yuecideng Nov 17, 2026
d887e99
fix(expansion): validate prepared scenes and PickUp path conflicts
yuecideng Nov 17, 2026
cba37a4
fix(expansion): retain reset holds and batch seeded solver paths
yuecideng Nov 17, 2026
4dad707
feat(expansion): reuse idle rows for bounded retry candidates
yuecideng Nov 17, 2026
db388b3
fix(expansion): retain owner-proven holds after terminal runtime
yuecideng Nov 17, 2026
9e29ad6
fix(expansion): keep speculative retry reuse opt-in
yuecideng Nov 17, 2026
86303c7
fix(expansion): compare effective scene preparation defaults
yuecideng Nov 17, 2026
5c557e8
feat(expansion): add workspace pose coverage and distribution reports
yuecideng Nov 18, 2026
910db94
feat(expansion): add bidirectional conditional layout sampling
yuecideng Nov 18, 2026
97493f3
fix(expansion): address point-IK and idle-row review feedback
yuecideng Nov 18, 2026
cfdc9f3
merge: sync main into expansion workloads
yuecideng Nov 18, 2026
5b5a307
docs: refresh scene expansion architecture evidence
yuecideng Nov 18, 2026
0557cdc
feat(expansion): support rolling layouts and measured phase coverage
yuecideng Nov 18, 2026
88c2e18
fix(packaging): include expansion distribution report template
yuecideng Nov 18, 2026
4132d86
docs(expansion): correct distribution API heading
yuecideng Nov 18, 2026
95990b8
merge: sync main into expansion workloads
yuecideng Nov 18, 2026
65f7d5d
feat(expansion): separate nominal scene and proposal seeds
yuecideng Nov 18, 2026
657daec
fix(expansion): stabilize installation identity in offline reports
yuecideng Nov 18, 2026
f18d7f7
fix(expansion): enforce task identity and layout admission contracts
yuecideng Nov 18, 2026
6dd084f
merge: integrate concurrent expansion updates
yuecideng Nov 18, 2026
f38aacb
fix(expansion): preserve admission deadlines for active rollouts
yuecideng Nov 18, 2026
c67c721
Merge remote-tracking branch 'origin/codex/expansion-workloads' into …
yuecideng Nov 18, 2026
37014e7
Merge main into codex/expansion-workloads
yuecideng Nov 19, 2026
0b2a018
docs: regenerate architecture snapshot after syncing main
yuecideng Nov 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion agent_context/MAP.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ topics:
- id: expansion-engine
title: Expansion Engine
aliases: [Expansion Engine, offline expansion, 扩增引擎, 批量专家采集]
keywords: [ExpansionEngine, ExpansionRunCfg, ExpansionJobCfg, ExpansionSession, ExpansionSessionCfg, ExpansionCoverage, EpisodeCoverage, GymExpansionHost, ArtifactEpisodeSink, MediaSpool, render_observation_variants]
keywords: [ExpansionEngine, ExpansionRunCfg, ExpansionJobCfg, ExpansionSession, ExpansionSessionCfg, ExpansionCoverage, EpisodeCoverage, GymExpansionHost, ArtifactEpisodeSink, MediaSpool, render_observation_variants, WorkloadProgramExecutor, WorkloadExecutionCfg, workload_count, GymExpansionHostExtensions, MeasuredExpansionValidator, MeasuredTrajectory, preview_expansion_comparison, 扩增虚影]
paths: [topics/expansion-engine/expansion-engine.md]
source_of_truth:
- embodichain/lab/expansion/
Expand Down Expand Up @@ -304,6 +304,7 @@ topics:
SimulationSceneExpansionHost, SimulationSceneInitialState, episode_prepare]
paths: [topics/scene-expansion/scene-expansion.md]
source_of_truth:
- embodichain/lab/_scene_config/
- embodichain/lab/sim/scene/feasibility/
- embodichain/lab/sim/scene/expansion/
- embodichain/lab/task_program/integrations/feasibility.py
Expand Down
10 changes: 10 additions & 0 deletions agent_context/topics/atomic-actions/atomic-actions.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,12 @@ Hosts acquire versioned measurements; verification is the boundary that commits
or invalidates observed effects. Preserve required stop/hold boundaries on the
executed control grid instead of relying only on native planner timing.

Row-local planning within a larger World must carry the same explicit
`context.env_ids` through FK/IK, root poses and diagnostics. HandOver applies this
to both source/destination arms, including its continuation and failed-waypoint
diagnostic routes. A one-row planning context does not implicitly select arena
zero or all arenas of the robot.

## Package map

| Concern | Source of truth |
Expand Down Expand Up @@ -99,6 +105,10 @@ are calibrated scene inputs. Preserve their selected owners and fingerprints;
never infer capabilities or general physical success from a sample name. The
host's sampled footprint/support checks remain finite measured acceptance,
separate from geometric proposal qualification.
`CoordinatedPlacement` retains each binding's configured grasp command during
approach/hold and on the support hand through release/retreat. Contact-blocked
measured jaw positions cannot replace these closing targets. Measured robot
observations and held-object transforms still supply geometry and validation.

## Recommended change sites

Expand Down
96 changes: 94 additions & 2 deletions agent_context/topics/expansion-engine/expansion-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,13 +10,21 @@ algorithms stay in `sim.motion.expansion`; scene and workspace algorithms stay i
| Closed configuration and preflight | `embodichain/lab/expansion/configured.py` |
| Canonical in-memory session, lifecycle limits and coverage port | `embodichain/lab/expansion/session.py`, `session_cfg.py`, `coverage.py` |
| Shared scheduling, admission and commit ports | `embodichain/lab/expansion/engine.py` |
| Rolling workload × B execution in one caller-owned World | `embodichain/lab/expansion/workloads.py` |
| Native workload World, source/evidence assembly and bounded export | `embodichain/lab/expansion/gym_workloads.py` |
| Declarative scene/motion jobs and pure configuration validation | `embodichain/lab/expansion/task_program_jobs.py`, `_task_program_config.py` |
| Owned result-preview launch, wait and stop | `embodichain/lab/expansion/preview.py` |
| Job assembly, stage recovery and inspect | `embodichain/lab/expansion/workflow.py`, `stages.py` |
| Native World ownership and compatibility facade | `embodichain/lab/expansion/host.py` |
| Task-owned render/source/transform/acceptance ports | `embodichain/lab/expansion/host_extensions.py` |
| Measured geometry, native substeps and sustained-contact acceptance | `embodichain/lab/expansion/acceptance.py` |
| Physics-free measured reference/ghost comparison | `embodichain/lab/expansion/comparison.py` |
| Native recipes, cohort preparation, step observation and episode assembly | `embodichain/lab/expansion/_gym_recipe.py`, `_gym_prepare.py`, `_gym_execution.py`, `_gym_frames.py`, `_gym_episode.py` |
| Durable recipes, attempts and invocation budgets | `embodichain/lab/expansion/journal.py` |
| Task Program execution with native or prototype motion | `embodichain/lab/expansion/program.py`, `real2sim.py` |
| Source-pinned sampling and joint scene candidates | `embodichain/lab/expansion/recipes.py`, `scene.py` |
| Artifact staging, checks and dataset export | `embodichain/lab/expansion/persistence.py`, `export.py` |
| Portable one-episode dataset export | `embodichain/lab/expansion/episode_export.py` |
| Native appearance and state rerender | `embodichain/lab/expansion/visual.py` |
| Bounded media blocks | `embodichain/data_pipeline/recording/media.py` |

Expand All @@ -38,7 +46,10 @@ configured jobs. `sampled_rigid_v1` collision profiles independently check all
physical box/plane scene bodies and task predicates from measured states. Their
robot-base transforms are rebound after each cohort's scene preparation.
State-only jobs retain independent and robot-mounted ContactSensors while
omitting cameras; contact acquisition does not require media capture.
omitting cameras; contact acquisition does not require media capture. Sustained
contact windows round off-grid durations upward, allowing only ULP-scale grid
roundoff. Their elapsed time sums measured physics intervals within the bounded
window; it does not subtract accumulated World clocks.
Immutable recipes declare
source/backend/phase/scope and shared-resource compatibility; the host checks them
before mutating the World.
Expand Down Expand Up @@ -71,6 +82,46 @@ sampling, whole-cohort preparation, active-row frame buffering and measured epis
assembly have separate owners. Preserve the host constructor, port methods and
existing readout properties used by benchmark/showcase instrumentation.

`GymExpansionHostExtensions` supplies non-owning callbacks for collection renderer,
scene source, immutable recipe transformation, measured validator and row plan
factory. `ExpansionRowContext` carries pinned identity without exposing mutable
host buffers. Hosts retain preparation, control clocks, T+1 capture, attachment
archival and confirmed receipts. Do not replace configured nominal sources with
an extension transform. `MeasuredExpansionValidator` presents the same lifecycle
for independent geometry and native contacts, including task-declared sustained
finger contact and drift limits. Every original physical substep remains required;
missing impulses, unknown actors and dropped samples reject acceptance.

`preview_expansion_comparison` consumes a caller-owned environment factory and
qualified measured episodes. Shared-time pause/switch controls retain the selected
time; physics stays disabled and render updates do not mutate saved evidence.
`MarkerFramePublisher` owns resident robot/object groups and publishes a coherent
cross-group frame through the public marker layer. Task assets and any collection
process orchestration remain outside the replay core.

`embodichain expansion preview --input` reads confirmed job outputs through `_preview_data.py`,
then replays full measured states using `_preview_render.py` and a loopback-only
`_preview_server.py`. Inputs retain their original evidence; no collection or
physics stepping occurs. It accepts one URDF robot and recorded rigid visual
objects (including file Mesh/USD via native visual loading), with fixed layouts;
unsupported geometry is rejected. Constant static backgrounds remain single
instances. Space filters group
input directories; missing motion annotations remain explicit. HTTP publishes a
single immutable image/state snapshot; WebRTC is follow-up work. FK and episode
budgets bound preload, controls are queued for the render owner, and native
resources/ports close on exit. See the example's `ghost-preview.md` for usage.

`preview --job` uses `_preview_job.py` to freeze lower/upper/center and stratified
interior parameter points, then invokes ordinary native collection in owned child
processes before opening the viewer. Physical ranges, atlas request domains and
explicit joint/via-point draws retain original validation. Each point enumerates
its configured motion/affordance recipes; exact-zero residuals collapse to the
unique nominal variant. Source semantics and aggregate attempts/time budgets are
checked, failed endpoints remain in the coverage manifest, and no replacement
sampling or success credit is allowed. Original job outputs remain untouched. Episodes archive `source_env_id` for
row-correct diagnostics even without a motion profile; ambiguous legacy grasp
annotations remain unavailable rather than selecting row zero.

`TaskProgramDemoBridge.program_id` and `env_ids` expose binding identity without
runtime traversal. `EmbodiedEnv.task_program_bridge_scope()` restores the borrowed
bridge even after failure. `Real2SimTabletopSource` and explicit
Expand All @@ -82,7 +133,7 @@ journal/duplicate-submission behavior.
Cohorts have fixed robot/joint topology, backend, clock, observation schema and
World appearance. Native scene-atlas input retains the complete layout/installation
candidate. Assets with different topology require separate deployment Worlds.
All rows reset and settle together. Only the owner steps physics, and row generators
The configured cohort path resets and settles its rows together. Only the owner steps physics, and row generators
acknowledge commands only after successful steps. Early termination must drain the
Task Program safe-stop handshake; completed rows stop contributing frames.
Native vector rejection cancels the row across semantic lanes and masks retained
Expand All @@ -91,6 +142,27 @@ post-policy observation clock. Expected Real2Sim acceptance rejection retains
measured checks and rejects only that row; source corruption and runtime errors
still stop collection.

`WorkloadProgramExecutor` is an optional rolling integration for custom native
sources. Its supplied environment must have exactly `workload_count * batch_size`
rows and be the sole live World in the process. Each workload owns quotas, local
candidate ordinals and admission budgets; its B-row lane is reused after that
group drains. Preparing rows settle through ordinary control steps while live
rows continue to record and validate every boundary. The source owns scene
proposals, bridge binding and all physical checks, using `ExpansionEngine.admit_row`,
`mark_row_started` and `complete_row` instead of editing private engine queues.
Only confirmed native receipts earn quota or permit reuse; duplicate identities
cannot earn credit twice. Clock spans must equal each trajectory's command count.
Compatible fixed-grid planning and pure solver requests can be cooperatively
merged on the owner thread. Scoped planning budgets retain per-row charges;
unsupported requests keep native execution. Export callbacks receive frozen
group evidence and never access the World. See [rolling execution](workloads.md).
Optional idle-row reuse keeps the same W×B capacity while competing retry
candidates use otherwise idle slots. Native write reservations still enforce
the original quota; peers are cancelled only after a confirmed receipt, using
measured partial evidence and the bridge's ordinary stop handshake. The low-level
source must supply an explicit cancellation port. Native Task Program jobs opt in
with the [idle-row reuse setting](workloads.md#source-ports-and-native-accounting).

Native numerical evidence has T submitted actions and T+1 measured states. It includes
all represented dynamic scene entities, source/action semantics, physical parameters
and call provenance. Position-velocity commands must use their ExpertActionSpec;
Expand All @@ -102,6 +174,11 @@ has bounded transient copies; no per-candidate dense full-episode image array is
Frame readers stream selected frames from stored blocks; full artifact audits still
validate bounded blocks. LeRobot export streams confirmed parents and visual children through the existing
recorder and checks the exact final journal identity set. ODS remains unchanged.
`export_expansion_episodes` additionally publishes one portable LeRobot dataset
per confirmed parent. Cached evidence reads still hash every file byte and
reconstruct checked, independently owned numerical tensors; source/receipt/media
inspection remains required. Immutable JSON nodes can be shared internally, while
public mutable projections own separate containers.

Pure visual replay requires complete scene state and an explicit observation-independent
expert contract. Physics is disabled across the entire World; render-state publication
Expand Down Expand Up @@ -130,3 +207,18 @@ measured state plus explicitly labeled FK, and pure-visual children retain their
physical parent's clock. Qualification requires admitted operator evidence, not
just a requested profile or completed nominal episodes. Report completeness and
capability acceptance are separate, including diagnostic unsupported combinations.

## Offline distribution reports

`python -m embodichain.lab.expansion._distribution` compares explicit before/after
confirmed job directories using the existing receipt inspector and episode loader.
It emits structured JSON/JSONL and a standalone HTML report with world-XY layout
bins, start/end joint counts, measured object paths, joint-motion summaries, and
existing geometry descriptor buckets. Version 2 adds full SO(3) orientation
features and recorded-URDF TCP FK with explicit gripper calibration. A lightweight
index drives filters; 100-record detail pages are shared by the offline viewer
and Agent JSON consumers. Matching task/robot/scene domains are required.
The report distinguishes cumulative additions from independent cohorts; observed
bins are not a feasible-volume certificate. The reader uses existing artifact
validation dependencies but creates no World. See
[distribution usage](../../../embodichain/lab/expansion/distribution.md).
Loading
Loading