Skip to content

Add portable mjfit assembly assets and precision demos - #981

Open
yuecideng wants to merge 29 commits into
mainfrom
codex/assembly-02-assets
Open

yuecideng wants to merge 29 commits into
mainfrom
codex/assembly-02-assets

Conversation

@yuecideng

@yuecideng yuecideng commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Description

Add a reusable precision-assembly toolkit with portable manifests, optional pinned mjfit generation, USD export and native compound/hinge preparation. Source body frames, visual/collision ownership and authored mass/COM/inertia remain explicit. Native preparation belongs to embodichain.toolkits.assembly.native, with prepare_native_configs as the mixed-body entry point.

Demos and benchmarks now select the Newton solver family explicitly. PrecisionRunCfg.newton_solver and demo --newton-solver accept mujoco_warp or dexuni; omission preserves the existing MuJoCo Warp selection. --solver continues to select the MuJoCo algorithm. The same iteration, capacity, integrator, CPU and explicit tolerance/MultiCCD settings are routed into direct MuJoCo Warp options or DexUni's nested mujoco_options.

Prepared results retain the actual Newton family, MuJoCo algorithm and SDK solver_diagnostics. These assembly profiles require DexUni's measured pure_mujoco dispatch; missing diagnostics, a mismatched family or a VBD/coupled dispatch is reported explicitly. Requested profiles and actual dispatch stay distinct in reports. Failed cells remain attempts of their requested profile, and each cell owns its preparation files. Independent reference cells are not duplicated by a Newton-family sweep. Constrained native S2–S4 probes expose the same family selection; passive-only overrides are rejected on constrained matrix calls.

Usage

embodichain run-example sim/assembly/visual_demo --backend newton --newton-solver dexuni --config circle --clearance .0001 --duration .2 --native-window --native-record --renderer fast-rt --output outputs/assembly/dexuni
python scripts/benchmark/assembly/precision.py --configs circle --backends newton --routes direct --newton-solvers mujoco_warp dexuni --duration .2

Programmatic callers use PrecisionRunCfg(backend="newton", newton_solver="dexuni") or run_native_probe(..., newton_solver="dexuni"). An explicit family is invalid on non-Newton backends. --use-mujoco-cpu remains an explicit single-world integration profile, never an automatic fallback.

Branch relationships

#981 remains the independent asset/toolkit PR on main. #982 is an independent RL runtime PR on main; #983 adds the temporal policy on #982. #984 adds assembly tasks on #983 and retains the complete asset implementation through ordinary merges from #981. This update propagates assembly fixes to #981 and #984 while preserving #982/#983's independent scope and concurrent RL fixes.

Type of change

  • New feature with correctness fixes, structural cleanup and documentation.

S2 accuracy and validation

The former constrained protocol forced one 1 ms integration step. Its stiff contact dynamics produced float32 transient wall errors up to 18.49 µm, exceeding the existing 10 µm budget. Native options below the library's float32 stopping floor do not eliminate the issue, so no SDK source patch or nominal-tolerance workaround is used.

ConstrainedProbeCfg.integration_substeps now defaults to 4 across CPU reference, raw MuJoCo Warp and native routes. A requested 1 ms interval uses four 0.25 ms physical steps. The physical duration, geometry, mass, force, damping and error budget are unchanged. Every settled physical step is scored; a 0.3 s trial scores 600 samples per direction. Reports expose requested/integration intervals, actual step counts and sample counts. --integration-substeps 1 retains the old protocol. This increases integration work and trace storage approximately fourfold.

The formal four-profile S2 CLI matrix uses the original circle asset, 100 µm clearance, four directions and 0.3 s:

Profile Worst wall error Unchanged budget Error check
MuJoCo CPU 1.19 µm 10 µm pass
Raw MuJoCo Warp 4.21 µm 10 µm pass
Newton/MuJoCo Warp 5.22 µm 10 µm pass
Newton/DexUni (pure_mujoco) 5.22 µm 10 µm pass

Native overall qualification remains unverified solely because live contact-overflow accounting is unavailable; the S2 error/finite/drift checks now pass. No metric or acceptance threshold was relaxed. The fixture retains the pinned generator's inline collision geometry/inertias and needs no generator at test time.

Isolated Python 3.12 / actual SDK 0.6.0rc2:

  • Focused assembly, benchmark, API-checker and context group: 150 passed, 4 GPU cases selected separately, 1 strict known SDK expected failure.
  • Real GPU regressions (legacy family/peg seating plus S2 accuracy): 4 passed. S2 asserts 1200 physical steps, 600 scored samples/direction, both families and the original 10 µm maximum-error bound.
  • Real wheel/source-free installed API/CLI and example packaging: 3 passed.
  • Matrix worker identity rejects a mismatched refinement profile. CPU and Warp references use the same integration interval. Existing full-window jitter/drift rejection is preserved.
  • Black 26.10.1 (black .), API coverage 4948/4948, context/whitespace and strict focused HTML checks passed. Existing cases are extended; the only added fixture closes the real GPU reproducer gap.

The previously qualified formal demo recordings (peg, blocked control, many pegs, M10 and CG gears) remain in the guide with provenance. Passive integration defaults are unchanged. No new training, model selection or full robot-policy replay is claimed.

Latest review resolution

All seven P2 review suggestions are covered:

Review concern Change and regression evidence
Architecture topic coverage Map assembly-assets to the toolkit architecture node, link its guide, refresh source evidence/snapshots.
Literal hinge names interpreted as regex Escape generated link selectors; gear[0] receives its own physics while decoy gear0 is untouched.
mesh_dir creates broken external references Reject this nonportable generation mode before writing any files; inline manifests remain the supported path.
NumPy configuration values break hashes/JSON Detach arrays/scalars into JSON primitives at import; real pinned-provider star generation and manifest roundtrip pass.
Hinge hash-directory symlink escapes caller root Preflight against the original resolved preparation root before writing; outside sentinel is preserved.
Thread lead oracle ignores scale Apply physical pitch scale, including fractional scale and multi-start regressions.
Constrained solver overrides silently ignored Existing explicit rejection is retained and tested in isolation, including CPU/algorithm options and refinement identity.

CI failure diagnosis and reproduction

The failing Documentation tests job reported that assembly-assets was registered in MAP but absent from architecture topic_ids. This is fixed in the curated node and generated snapshot. The exact frontend/Sphinx check sequence passes locally: 17 data/state/link tests, 8 browser tests, 10 real Sphinx/browser integration tests, plus format checking. Native USD comparison tests also register SDK schemas before their first stage, removing collection-order dependence while leaving portable export SDK-independent.

#982's reported offline/export jobs pass; its SDK jobs were queued, not failing. #983 had no reported checks at inspection. Their independent runtime/policy smoke groups pass 230 and 39 tests respectively. No assembly code is reintroduced into either branch. The self-hosted #981 build/test jobs remain queued.

The new Documentation tests run on 4ad223908 completed successfully, including the formerly failing architecture coverage, browser and real Sphinx integration steps. The new lint, documentation-change and test-plan jobs also pass. Queued self-hosted work is not reported as completed.

Documentation and research

docs/source/features/toolkits/precision_assembly.md is linked from Toolkits, API reference and the example README. It covers functional entry points, installation, generation/manifests, native preparation, explicit solver choices, matrix commands and measured limits. Five embedded H.264 clips, posters, before/after S2 measurements and versioned provenance ship under docs/source/_static/assembly/ (about 0.5 MB). The DexUni clip now comes from the formal CLI, with source hashes and SDK dispatch metadata, rather than an experimental host.

The guide credits Blind Precise Assembly, links the authors' project and pinned mjfit source, and includes BibTeX. It distinguishes asset/passive-demo integration from the paper's learned-policy and sim-to-real results.

Dependencies and limits

The optional mjfit provider is pinned to 5833f465896e1e06bc4be4f003be847f5183a4db. Prepared manifests load without mjfit or MuJoCo. Normal installation delegates compatible simulation dependencies to SDK/Newton; there is no assembly extra or duplicate project-level MuJoCo constraint. No dependency patch files, overlay scripts, dedicated patch tests or patch-specific packaging hooks are introduced.

SDK rc1/rc2 static USD import drops child colliders for disabled compound bodies; the regression remains a strict version-specific expected failure and direct construction preserves the pieces. FastRT is the measured native recording profile; Hybrid recording crashed under the remote Xvfb setup. The failed documentation workflow is reproduced and fixed as detailed above; whole-site rendering can still emit unrelated existing warnings. The new guide passes its strict focused HTML build.

Checklist

  • Format with Black 26.10.1 and check whitespace.
  • Update affected guide, API entries and project context.
  • Verify focused CPU/GPU coverage and test necessity.
  • Build and consume the real wheel outside source.
  • Run formal demo and benchmark entry points; retain failed/unverified measurements honestly.
  • Preserve stack ownership and synchronize dependent PRs.

@yuecideng yuecideng added toolkit Standalone robotics tools, including GraspKit and URDF assembly. assets Robot and scene assets, meshes, materials, asset catalogs and downloads. benchmark Benchmark protocols, suites, metrics, evaluation tools and technical reports. feature New user-facing capabilities. labels Nov 4, 2026
@yuecideng
yuecideng marked this pull request as ready for review November 9, 2026 12:22
Base automatically changed from codex/assembly-01-simulation to main November 15, 2026 06:23
@yuecideng
yuecideng requested a review from han-xudong November 15, 2026 08:54

@han-xudong han-xudong left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed PR #981 at 39b47ac9f71da99fa93b3f3f2fa1f188edd7f249 against b629c7158b8180934c0990b71e12e1031ddac16e.

Seven P2 findings are attached inline. I recommend addressing them before merging.

Validation included focused benchmark checks (34 passed, 1 skipped), native configuration checks (22 passed; one failure was due to the local older SDK missing ConstraintDesc), API/context checks (12 passed), and targeted reproductions of the reported cases. The current documentation CI failure was also checked. Native rc2 simulation, GPU/rendering, and a fresh wheel build were not rerun.

Comment thread agent_context/MAP.yaml
related_topics: [simulation-system, data-assets, env-framework, task-engine]
related_topics: [simulation-system, data-assets, env-framework, task-engine, assembly-assets]
status: active
- id: assembly-assets

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Add the new topic to the architecture graph

Registering assembly-assets here without a corresponding topic_ids entry in docs/architecture/curated.json breaks the existing architecture coverage contract. docs/architecture/web/tests/data.test.ts requires every registered topic to be represented, and the current Documentation tests job fails specifically because assembly-assets is missing. Add the topic to an appropriate architecture node/view alongside this registration so the documentation checks pass.

Comment thread embodichain/toolkits/assembly/_urdf.py Outdated
asset_physics_mode="overlay",
link_attrs={
"moving_part": LinkPhysicsOverrideCfg(
link_names_expr=[body.name], attrs=properties

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Escape literal body names before using them as link selectors

link_names_expr is interpreted with re.fullmatch, but body.name is a literal source identifier. A valid world-mounted MJCF hinge named gear[0] imports and passes this exporter's name validation, yet the generated selector ['gear[0]'] cannot match the actual link gear[0]. The subsequent configure_articulation_desc() call therefore raises ValueError: Not all regular expressions are matched, preventing native preparation. Escape the source identifier, e.g. with re.escape(body.name), before constructing the selector.

# Include standalone solver options for the reference. The adapter discards
# worldbody decorations and floor, retaining only named physical bodies.
xml = mjfit.build(cfg, standalone=True)
asset = import_mjcf(xml, config=cfg, seed=seed, generator_revision=revision)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Resolve generated mesh paths before compiling the MJCF string

The public generator's mesh_dir option is forwarded here, but its external mesh paths are then compiled without their base directory. With the pinned mjfit revision, build_assembly("circle", mesh_dir="/tmp/<temporary-directory>/meshes") writes 35 OBJ files and then fails with Error opening file 'meshes/plate_visual.obj': the generated XML uses a relative compiler meshdir, while import_mjcf() compiles the string relative to the process working directory. Resolve/embed those generated assets while retaining a portable source representation, or explicitly reject this option before generating files.

return AssemblyAsset(
generator_revision=generator_revision,
source_mjcf=xml,
config=dict(config or {}),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Normalize public shape arrays before storing manifest configuration

Copying the generator configuration verbatim can retain NumPy values that the manifest's JSON encoder cannot serialize. For example, build_assembly({"kind": "peg", "hole": mjfit.shapes.star(5, .012, .006), "clearance": 1e-4}) successfully creates two bodies using the pinned provider, but asset.content_hash then raises TypeError: Object of type ndarray is not JSON serializable. Consequently caching, saving the manifest, USD export, and preparation paths that need the hash fail for this public programmatic configuration. Normalize supported arrays/scalars to stable JSON primitives before retaining the configuration.

Comment thread embodichain/toolkits/assembly/native.py Outdated
configs.update(
to_articulation_configs(
asset,
Path(directory) / asset.content_hash,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Check the hinge cache subdirectory against the caller's root

If <directory>/<asset.content_hash> already exists as a symlink to another directory, the default prepare_native_configs(..., file_visuals=False) follows it and writes generated hinge files outside the caller's preparation directory. to_articulation_configs() first resolves this derived path and treats the resolved destination as its trusted root, so its later containment checks no longer protect the original root. This is reproducible with an imported hinge and can create or replace same-named URDF/OBJ/MTL files outside the cache; the rigid visual writer rejects the analogous escape. Validate the resolved hash subdirectory against the caller's resolved root before delegation.

* round(cfg.duration / cfg.timestep)
* cfg.timestep
/ (2 * math.pi)
* float(asset.config["pitch"])

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Include asset scale in the thread lead oracle

The pinned mjfit generator multiplies every thread length, including pitch, by scale, but this ideal-advance calculation uses the unscaled configuration pitch. For build_assembly("m10", scale=2) with duration 0.2, timestep 0.001, and spin rate 15, the correct physical advance is 0.001432394487827058 m. Supplying exactly that advance to _score() produces an ideal advance of 0.000716197243913529 m, approximately 100% relative error, and success rate 0. Include asset.config.get("scale", 1.0) in the physical pitch so correctly scaled motion is not rejected.

run_warp_probe,
)

selected = ConstrainedProbeCfg(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Apply or reject solver overrides for constrained probes

For S0/S2/S3/S4, this conversion drops cfg.solver and cfg.use_mujoco_cpu, even though the CLI accepts both and the original values are later retained in the result's config. In a reproduction requesting solver="pgs", the output records PGS while the prepared reference model still uses mjSOL_NEWTON; similarly, the native constrained runner does not select the requested Newton CPU route. The passive-only guard rejects tolerance/MultiCCD overrides but does not reject these options. Propagate them into the constrained execution path, or reject them explicitly before launching a cell, so recorded experiment settings match actual execution.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

assets Robot and scene assets, meshes, materials, asset catalogs and downloads. benchmark Benchmark protocols, suites, metrics, evaluation tools and technical reports. feature New user-facing capabilities. toolkit Standalone robotics tools, including GraspKit and URDF assembly.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants