Skip to content

Standardize expansion host, acceptance and ghost comparison - #1192

Open
yuecideng wants to merge 11 commits into
mainfrom
codex/expansion-workloads
Open

yuecideng wants to merge 11 commits into
mainfrom
codex/expansion-workloads

Conversation

@yuecideng

@yuecideng yuecideng commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

Description

Downstream manipulation expansion currently has to replace motion generation globally, override private host startup/execution, and reach into marker internals for ghost playback. Add native interfaces while keeping task assets, calibration, sampling ranges and physical thresholds task-owned.

  • Explicit MotionPolicy.phase_paths and MotionGenOptions.waypoint_modes compose free-space joint paths and Cartesian contact phases on the original grid. Ordinary callers retain their semantics, failure masks and native recovery.

  • GymExpansionHostExtensions exposes collection rendering, scene proposals, recipe transforms, measured validators and row plan factories. The host retains preparation, clocks, T+1 capture, attachment archival and persistence. TaskProgramSameGridExpansionFactory.from_native_plan reuses existing operators and Atomic reconstruction for measured-plan affordance/trajectory adaptation.

  • MeasuredExpansionValidator presents one lifecycle for independent geometry and full native contacts, including sustained finger contact and drift checks. Unknown actors, missing impulses and dropped evidence fail closed. Tasks supply permissions and thresholds.

  • MeasuredTrajectory, the reference recorder and preview_expansion_comparison provide shared-time pause/switch controls and physics-free solid/ghost playback. MarkerFramePublisher retains resources and publishes visible robot/object groups together using actual visibility. Playback uses the native render-state owner and restores the borrowed scene and physics state.

  • pose_nms_scope supplies nested context-local memory budgets without replacing backend functions. Explicit call arguments override the scope; selection arithmetic/order remain unchanged.

  • Retain the opt-in single-process/single-World workload_count × batch_size executor, rolling refill and native session/receipt accounting, compatible planning/acceptance batching, CUDA fixes, owned snapshots, content-verified episode reads and portable per-episode LeRobot export.

  • embodichain expansion preview --job JOB freezes parameter lower/upper/center and stratified interior points, runs the native Task Program/acceptance pipeline, then automatically opens the full-model ghost viewer. Physical intervals, scene-atlas request domains and explicit signed joint/via-point draws keep original limits and validation; finite sampling does not claim continuous or all-corner coverage. Failed/unrun/reused/fallback cases remain explicit under aggregate budgets. --input (alias --inpu) replays existing job/preview results without collection; job-mode --output owns a new directory. Native rendering uses loopback HTTP JPEG; WebRTC remains TODO.

Configured/cohort defaults remain compatible. Rolling execution is an explicit WorkloadSource integration; this does not switch the configured CLI scheduler automatically. No downstream tasks, private assets, calibrated poses or workflow processes are included. Dependency: greenlet>=3.2,<4; DexSim pin unchanged.

Validation

  • Result-preview integration: 339 focused configuration/operator/Task Program/episode/preview/context tests passed. Native end-to-end distribution verification used 13 varying dimensions and center/lower/upper/interior points: all 14 unique trajectories passed unchanged acceptance, without trajectory fallback or reused affordance branches. Browser filtering 14→4→2→1→0, coverage reporting and zero-physics replay passed. A no-motion-profile job also collected and opened successfully; row identity is now archived and ambiguous legacy diagnostics stay unavailable. The default example preflights 30 unique attempts. Black, API docs, context, JavaScript and link checks passed; verification services closed.

  • Final VCS-built installed wheel (ab04d972b5b75042434498db9a889b4edd03b6fe): all five original downstream tasks completed 10/10, 50/50 total, under their original attempt/time/physical acceptance budgets. A task-owned thin adapter retains the original process scheduler and task parameters while using the new public phase/host/acceptance/transform/export/NMS ports. Both old global-motion installation sites and the NMS function replacement are removed from that adapter.

  • Independently audited all 50 accepted episodes: native receipts/content hashes, real LeRobot SDK round trips, T+1, protected samples, scene ranges, motion/FK limits, measured terminal/contact criteria and the original 71-sample bimanual hold/drift requirement. Parameters match the original benchmark inputs byte-for-byte. Runs use PYTHONHASHSEED=0 and the original CPU physics configurations; this is not five-task GPU qualification or a throughput comparison.

  • All five real X11 viewers passed pause, same-time selection, resume, exit, zero playback physics steps and scene/target restoration. Four use the preceding feature wheel and bottle relay uses the final wheel; final comparison/publisher/renderer bytes are identical. Screenshots are real rendered frames. The native SDK emits Vulkan present validation warnings under Xvfb; picture, input and restoration were checked independently.

  • Final installed-wheel real four-row rolling execution passed safe rejection/hold, refill, queued groups and four confirmed receipts in one World. Real two-row tests passed explicit phase parity, ordinary-call preservation and selected-row acceptance isolation.

  • 386 affected-module tests passed (2 skipped, 7 deselected); 36 NMS checks passed (4 deselected), including real CPU/GPU behavior and context isolation. Reused existing coverage and added only measured row/phase, comparison ownership/identity and NMS scope coverage.

  • 53 documentation/context tests passed; all 4,942 public exports are documented. Sphinx dummy succeeds with the same existing 888 warnings as the base; normalized warning sets are unchanged. Repository-pinned Black and whitespace checks pass.

  • Incomplete intermediate runs remain separate, including an earlier feature-wheel handover quota failure. The final five-task matrix met every quota; this does not establish repeated-run determinism, full-suite coverage or a universal throughput improvement. Remote CI remains subject to runner availability.

Type of change

  • Enhancement (non-breaking change which improves existing functionality)

Screenshots

Native-window screenshots and interaction evidence are retained in the local qualification report. Private task assets/data are not included in this repository.

Checklist

  • Ran repository-pinned Black 26.10.1 before each commit.
  • Reviewed and updated affected API documentation and agent context.
  • Verified public API coverage with python docs/scripts/check_api_docs.py.
  • Selected proportional behavioral checks and reviewed added/extended test necessity.
  • Reviewed dependencies and native ownership boundaries.

@yuecideng yuecideng added enhancement Improvements to existing capabilities; use feature for new capabilities. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. kinematics FK, IK, Jacobians, reachability and kinematic solvers; not physics solvers. motion gen Robot motion planning, MotionGenerator, trajectory generation, retiming and expansion. dependencies Dependency versions, upgrades and compatibility. sensor Sensor models, attachment, lifecycle, extension interfaces and observation contracts. data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. task-program Task Program language, semantics, compiler, runtime and integration contracts. labels Nov 14, 2026
@yuecideng
yuecideng marked this pull request as ready for review November 14, 2026 09:51
Order permission queries after the latest sensor sample on the caller's CUDA stream without resampling or advancing histories. Cover pending forbidden contacts and dropped evidence in standalone and history sampling paths.

Validation: 75 targeted tests passed; Black 26.10.1, public API documentation coverage and agent context checks passed.
@yuecideng yuecideng changed the title Add single-World workload execution and batch checks Standardize expansion host, acceptance and ghost comparison Nov 14, 2026
Share Default CPU numeric observations within bounded command-gather scopes, reconstruct immutable standard targets, and transfer fresh internal command ownership while preserving public snapshots and validation.

Keep native multi-chunk grasp geometry and reductions resident on CUDA within each query, preserving samples, thresholds, stream ordering and independent results. Add focused regression coverage and a reproducible runtime comparison driver.

Validation: 201 focused tests including CUDA and 42 context tests passed; Black 26.10.1, API docs 4924/4924 and diff checks passed. Sources match the final benchmark manifest.

Forward/reverse sensor-free UR5 execution versus main 0119171 (seconds): small main 10.34-10.45, off 9.07-9.31, on 8.31-9.08; medium main 49.24-57.12, off 47.19-50.90, on 48.66-49.57; large main 251.77-260.14, off 189.57-196.42, on 192.07-200.29. All 492 program attempts accepted. Default CPU physics retains legacy CUDA grasp classification; these shared-host single-seed results are not physical qualification.
Restore the existing Default CUDA backend policy. Remove the newly introduced public selector and its ordinary-reporting test variant, retaining CPU contact permission coverage and documenting Direct GPU static-identity limits.

Validation: 53 configuration/contact tests including CUDA and 42 context tests passed. Black 26.10.1, API docs and diff checks passed.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

data-pipeline Data collection, ODS, sampling, recording, persistence, conversion and analysis. dependencies Dependency versions, upgrades and compatibility. enhancement Improvements to existing capabilities; use feature for new capabilities. kinematics FK, IK, Jacobians, reachability and kinematic solvers; not physics solvers. motion gen Robot motion planning, MotionGenerator, trajectory generation, retiming and expansion. physics Physics backends, dynamics, contacts, fluids, deformables and physics solvers. sensor Sensor models, attachment, lifecycle, extension interfaces and observation contracts. task-program Task Program language, semantics, compiler, runtime and integration contracts.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant