Skip to content

Support multi-environment scene expansion without legacy APIs - #1206

Open
yuecideng wants to merge 3 commits into
mainfrom
codex/scene-expansion-multi-env
Open

yuecideng wants to merge 3 commits into
mainfrom
codex/scene-expansion-multi-env

Conversation

@yuecideng

@yuecideng yuecideng commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

Description

The scene preparation host and Task Program attempt API currently reject Worlds with more than one environment. This change replaces that single-row contract with explicit environment-to-candidate/result mappings, so selected environments can be prepared, restored and evaluated together while retaining independent recording eligibility.

execute_scene_variants(env, {3: candidate_a, 1: candidate_b}, ...) executes only rows whose initial checks pass. Program completion and measured acceptance use each row's evidence, so one rejected candidate cannot invalidate a successful peer. The host preflights every requested row before reset/writes, preserves unselected state during preparation/restoration, and rejects snapshots assigned to another row. Vector bridges isolate policy sessions, and one scene attempt owns the World at a time.

All environments share one physical clock. Settling requires the complete World selection and runs once for the batch. Source tracing and specialized workspace/search adapters retain their own qualification limits.

Refs #760. The remaining general-region and large-scene roadmap is outside this PR.

Dependencies: no new dependencies.

Breaking API changes

  • Replace execute_scene_variant with execute_scene_variants; preparation returns the requested row mapping, and measured validators accept an environment ID.
  • SimulationSceneExpansionHost accepts ordered env_ids and row-addressed initial validation. Use initial_states, capture_initial_states(parent_scene_ids=...) and restore_initial_states(...) mappings.
  • SceneVariantEpisodeResult requires env_id, evaluates row-specific completion, and publishes metadata schema version 2.
  • Remove the Task Program workspace compatibility module. Import the shared service from embodichain.lab.sim.scene.workspace, whose result is SceneCheckReport.

Production feasibility, Real2Sim collection, asset-generalization consumers, examples and tests are migrated directly; the old signatures and compatibility imports are not retained.

Validation

  • Black 26.10.1: black .; syntax, Apache headers, future annotations and public module exports checked for all 14 changed Python files.
  • Public API coverage: 4,912/4,912 exports documented; agent-context structure and affected navigation checked.
  • 543 CPU checks passed across scene state/operations, Task Program integration, Real2Sim collection, asset generalization, Expansion Engine and context tooling.
  • 58 API-documentation and architecture-tooling checks passed.
  • Native scene-expansion qualification: 4 isolated no-render checks passed (pytest -q --noconftest -c /dev/null tests/gym/envs/task_program/test_scene_expansion_sim.py): Default/Newton two-row execution, Default four-row partial selection (3, 1), and single-row recording/restoration.
  • Full Sphinx dummy build passed at the committed revision (exit 0; 874 repository documentation warnings). The updated scene API pages also passed a focused dummy build. The architecture source-evidence generator now completes against the committed source.

New coverage closes the multi-row integration gaps: noncontiguous selections, all-row preflight before mutation, snapshot row ownership, independent initial/program/measured rejection, delayed persistence and nested-World ownership. Existing production suites retain single-row/reset/receipt behavior. Tests dedicated solely to the removed workspace compatibility adapter are deleted; shared workspace IK/FK coverage remains at its source owner.

Affected scene-expansion and environment-lifecycle context and API references are updated. Other affected context remains accurate because task semantics, registration and persistence ownership are unchanged.

Runnable scene construction, solving and augmentation

embodichain run-example expansion/scene_layout --output /tmp/scene-layout

The native scene-only example incrementally declares a table, two namespaced containers, a visual keep-out marker and six colored parts. It repairs the deliberately invalid authored layout with solve_layout, generates three addressed generate(recipe, seed, ordinal) candidates under identical assets/constraints, applies their poses through scene services, then qualifies actual settled poses with check_layout and measured convergence. Outputs include native before/after RGB, a four-layout gallery, a construction/solving/augmentation video, an HTML report and JSON plans/evidence.

A real run passed: 2 overlapping part pairs, 2 keep-out intrusions and 1 boundary violation became zero in the repaired layout and all three variants. Fixed components stayed unchanged; geometry operations did not step physics; gravity converged after 51 physical steps (0.51 seconds). Variant RMS XY displacements from the repaired layout were 0.61/0.67/0.71 m. The example explicitly limits its claims to the configured XY/AABB constraints and native settling checks.

--help, Black, API coverage and 82 existing example-launcher/composition/layout/procedural checks passed. This follow-up changes only example code and instructions; the production contract coverage above remains applicable.

Type of change

  • Breaking change (existing functionality requires caller migration)

Checklist

  • I have run black . to format the code base.
  • I reviewed affected documentation and agent context, and updated the changed contracts and navigation.
  • Public API changes are reflected in the API docs (python docs/scripts/check_api_docs.py).
  • I verified behavioral coverage and reviewed the necessity of added, extended and removed tests.
  • Dependencies reviewed; no updates required.

Use explicit row mappings for scene preparation, restoration and Task Program
attempts. Keep initial checks, canonical completion, measured acceptance and
recording eligibility independent per environment. Migrate production callers
and remove the workspace compatibility adapter.

BREAKING CHANGE: use execute_scene_variants and the host's plural state APIs;
validators receive environment IDs and results require an explicit env_id.
Refs #760.
@yuecideng yuecideng added enhancement Improvements to existing capabilities; use feature for new capabilities. breaking User-visible incompatible changes requiring migration. gym Environment lifecycle, managers, configuration composition and registration. task-program Task Program language, semantics, compiler, runtime and integration contracts. scene Shared scene composition, layout, state restoration, validation and expansion services. labels Nov 15, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

breaking User-visible incompatible changes requiring migration. enhancement Improvements to existing capabilities; use feature for new capabilities. gym Environment lifecycle, managers, configuration composition and registration. scene Shared scene composition, layout, state restoration, validation and expansion services. task-program Task Program language, semantics, compiler, runtime and integration contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant