[han-xudong]

Description

Adds opt-in TactileIndentationCfg.shear for per-cell compressive load, two-axis shear force and elastic displacement. Ordered probe indices expose sparse marker displacements. Relative pad motion against the winning shape drives the elastic state, with a Coulomb slip limit and row-selective reset. The Warp update reuses device buffers on flat, spherical-cap and explicit layouts.

The penalty model estimates pad response from clipped geometric depth, material stiffness and friction. Its fields are independent of solver contact forces.

Refs #1174.

Stack

• Layer 16/17 of sensor extensions stack #1186.
• Base branch codex/sensors-4e-rgb.
• Depends on #1196
• Followed by #1207

Type of change

• New feature

Review focus

Review relative target motion, Coulomb slip, contact changes, per-row reset, and sparse marker indexing. The geometric penalty estimate is independent of measured solver contact impulse.

Validation

The combined CPU/CUDA regression selection passed 88 tests on DexSim 0.6.0rc1. Regressions cover a transient pose change between completed steps, unchanged data and metadata after direct update(), and accurate nonzero marker motion and shear with strided mount and target matrices on both devices. The same 88 tests passed on DexSim 0.6.0rc2.

Current CI checks

The step-56 render and modeled shear field are synchronized. Marker arrows are scaled 2× for visibility, and the curves show the original values.

Public sensor recordings

In four synchronized GelSight Mini recordings, 40–63 physical markers per clip passed tracking checks. At the measured lateral-force peaks (2.00–8.84 N), p90 marker motion was 3.15–8.08 px. Spearman correlations across the four clips ranged from -0.56 to 0.66. In the 81-step simulations, X shear reversed with the commanded slide and cleared after release. The flat profile exposed sparse markers and the curved DIGIT 360 profile exercised the marker-free path. The DIGIT 360 clips lack synchronized independent shear labels. Force magnitude and cross-device accuracy remain uncalibrated.

TaF source · TaF MCAP conversion · Sparsh-X · DIGIT 360 CAD

Held-out nominal endpoint replay

GenForce provides paired Nano17 readings and marker masks at four phase endpoints. Replay reconstructs the published UR5e commands. Gains and the camera-axis alignment were fitted using eight sphere trials, then frozen for eight sphere_s trials (32 endpoints, 43 tracked markers per trial). Shear magnitude MAE was 0.103 N versus 0.208 N for zero shear. At maximum lateral excursion it was 0.199 N. Mean marker vector error was 1.52 px versus 1.62 px for zero displacement. Pixel conversion uses the nominal camera scale. The release lacks framewise measured XYZ and synchronized raw RGB for these trials. The comparison is at nominal endpoints and does not qualify measured-trajectory or slip-timing accuracy.

GenForce dataset · Published UR5e protocol

Architecture. The completed-step scheduler owns all field publication and shear history. A direct update() retains the latest sample. Warp kernels live under compute/geometry/_warp/. Depth and shear share the same mount and target matrices, with strided inputs made contiguous once. The model uses generic ray geometry and runs independently of the DexSim hydroelastic snapshot. Sensor roadmap #1174 · Extension contract #835

Performance

Full-step regression

RGB and shear enabled at full contact on the RTX 5070 using DexSim 0.6.0rc2 and Newton MuJoCo Warp. Timings include pose and velocity writes, one no-render simulation step and sensor acquisition. The table reports the mean of the two run-level percentiles, using forward and reverse environment and variant order. Each pass used 25 warmup and 100 measured steps per variant.

Environments	Before p50 / p95 (ms)	Updated p50 / p95 (ms)
1	7.77 / 9.00	7.39 / 8.45
512	8.16 / 9.72	7.57 / 8.58
2,048	10.23 / 12.04	9.96 / 11.79

Both passes verified matching depth, hit masks and shape IDs across variants. The contact-grid and depth control also varied between runs. These are full-step measurements.

At 2,048 environments, the two 100-step passes changed p95 by -20.3% and +16.4%. A repeat with 100 warmup and 250 measured steps yielded 10.25 / 12.16 to 10.22 / 12.07 ms p50 / p95 with RGB and shear. The contact-grid and depth control moved from 10.36 / 12.70 to 10.50 / 12.12 ms. Results are sensitive to run order and system timing.

CUDA target pose preparation

A single Warp kernel converts packed rigid positions and xyzw quaternions into matrices for CUDA ray targets. The ray provider continues to resolve mesh offsets and topology. CPU pose reads follow the existing path. Nsight at 2,048 environments recorded 51 to 9 GPU operations per target pose read and 524 to 411 µs host p50.

The paired replay below alternates the old and new conversion in the same Newton simulation, with 240 measured complete steps per path and scene. It verifies depth, shape IDs, RGB, forces, and marker outputs against the same motion. Other CUDA workloads were present during timing.

Environments	Contact fraction	Before p50 / p95 (ms)	Updated p50 / p95 (ms)
1	0%	9.52 / 12.66	9.23 / 12.12
1	100%	9.29 / 10.71	8.94 / 10.24
512	25%	10.56 / 12.85	10.30 / 11.55
512	100%	10.51 / 19.25	10.21 / 11.79
2,048	25%	12.44 / 17.41	11.96 / 14.86
2,048	100%	12.33 / 14.03	11.84 / 13.54

The joint CPU/CUDA regression selection passed 93 tests on each of DexSim 0.6.0rc1 and 0.6.0rc2. The ray provider regression covers translated and rotated rigid targets, authored and cooked geometry, and topology invalidation. A separate cross-check of 4,096 strided poses on a non-default CUDA stream found a maximum matrix difference of 4.77e-7 against the existing Torch conversion.

Original feature benchmark

RTX 5070 and DexSim 0.6.0rc2. The full-step benchmark uses an 8×8 contact grid and 16×16 curved indentation pad per arena. Timings include pose and velocity writes, one full no-render Newton MuJoCo Warp step and sensor acquisition, with CUDA Graph disabled. Each variant used 20 warmup and 60 measured steps. The benchmark checks base depth parity and the active RGB and load rows.

Environments at full contact	Baseline p50/p95 (ms)	RGB p50/p95 (ms)	Shear p50/p95 (ms)	Both p50/p95 (ms)
1	7.676 / 8.715	7.427 / 8.330	7.329 / 8.675	7.934 / 9.468
128	7.843 / 8.610	7.826 / 8.609	8.139 / 9.121	7.929 / 8.635
512	8.970 / 9.899	8.920 / 9.988	8.668 / 11.326	8.952 / 10.309
2,048	10.555 / 12.300	10.585 / 12.352	10.697 / 12.335	10.685 / 12.480

At 2,048 arenas with 25% contact, baseline and both-enabled p50 were 10.510 and 10.464 ms. Run python -m scripts.benchmark.sensors.tactile_step --envs 1 128 512 2048 --fractions 0.25 1 --variants both-dense both-rgb both-shear both-rgb-shear --warmup 20 --repeats 60.

A reverse-order pass at 2,048 arenas also matched the depth results. Other GPU workloads ran during this repeat and p50 ranged from 23 to 41 ms across variants. The full-step readings do not isolate incremental sensor cost under contention.

The first-run full-step comparison covers both RGB and shear. The lower chart shows the increase relative to contact grid plus indentation at each environment count.

Scope limits

The output represents an elastic penalty approximation. Real material and marker calibration still need recorded physical contact trajectories.

Checklist

• Code formatting checked.
• Affected documentation and project context reviewed.
• Public API documentation coverage checked.
• Focused behavioral coverage verified.