Bench2Dex benchmarks bimanual robots with simulated touch
Bench2Dex connects teleoperation, replay, tactile maps, and evaluation across 12 hand embodiments. We examine the data contract and distinguish simulation access from physical sensor validation.
Benchmark and resource review — September 14, 2026 preprint; access checked September 18, 2026
Bench2Dex is a simulation benchmark for long-horizon, two-handed robot manipulation. Its preprint describes 12 dexterous hand embodiments, 26 task–embodiment settings, and about 1,300 human-teleoperated demonstrations. Its contribution is a shared pipeline for collecting actions, replaying multimodal observations, and evaluating task completion under controlled scene changes.
These demonstrations are collected in simulation. They are not recordings from 12 physical robot hands. The tactile maps describe local contact geometry and do not reproduce the optical response of a particular DIGIT or GelSight sensor. That makes Bench2Dex useful for studying data interfaces and simulated manipulation, with a clear boundary around claims about physical touch.
From teleoperation to offline observations
The collection setup uses a Manus glove for hand motion and an ARKit wrist-tracking stream for arm targets. Hand keypoints are retargeted to the chosen robot hand; arm targets pass through inverse kinematics. The pipeline records the commanded action and resulting post-step state. This ordering matters when aligning an action with the observation it caused.
The paper separates lightweight online motion recording from offline replay that renders richer observations. Its full schema includes RGB, depth, joint state, object state, tactile maps, bounding boxes, and occupancy labels. These are supported modalities, not a guarantee that every downloaded episode contains every field. The public README notes that depth is omitted in its current storage workflow because of size. Inspect the episode’s modality metadata and actual datasets before designing a loader. Official repository.
Reading public files does not require owning a robot or tactile sensor. Collecting new demonstrations with the documented input setup requires the corresponding tracking equipment, and running the simulation requires a supported NVIDIA GPU/software environment. Our robot data collection guide explains the separate roles of capture, synchronization, quality review, and training export.
What the tactile map actually measures
Bench2Dex reconstructs contact surfaces for each supported hand and stores local surface points and normals. During replay, rays cast against task-object meshes produce a contact-consistency-filtered depth signal. The pipeline encodes that signal into an 8-bit image-like tactile map and also defines raw metric ray depth and a validity mask. The paper describes 240 × 240 tactile maps in the released-data schema. Section 3.3 and Appendices C–D.
The compact image values run from zero to 255. They are not newtons, kilopascals, or camera intensities measured through an elastomer. Raw ray depth has metre units, but it remains simulator geometry rather than a force estimate. A common map shape also does not make the different hands mechanically equivalent.
The main four-policy table should not be read as a tactile-input ablation. The paper describes ACT and Diffusion Policy using multi-view RGB and joint state, alongside fine-tuned π0.5 and GR00T N1.5 baselines. Availability of tactile observations does not by itself prove a performance gain from using them. A matched tactile-enabled versus tactile-disabled experiment is needed for that claim.
An HDF5 contract worth inspecting before training
Appendix D defines a full per-episode HDF5 organization. The following is a reading guide to that schema, not an assertion that RoboSkin downloaded and validated the complete release.
| Field or group | Meaning | Check before using it |
|---|---|---|
| meta/ | Task, robot, modality list, schema and collection metadata | Confirm the embodiment and available modalities |
| time/frame_index, time/timestamp_ns, time/sim_step | Frame order, stored timestamp, and simulation step | Check order and alignment; timestamp units alone do not establish wall-clock latency |
| action/ and robot/qpos | Commanded action and observed joint positions | Match joint names, dimensions, control mode, and action/observation ordering |
| frame_valid and frame_errors | Capture validity and error information | Filter or investigate invalid frames; do not silently fill missing observations with zero |
| robot/tactile/tacmap/{site_name} | Per-site uint8 tactile image | Check site registry, dimensions, and encoding |
| robot/tactile/distance_along_normal_m and robot/tactile/contact_mask | Raw ray depth and contact-validity datasets | Keep geometric units and validity separate from the compact image |
| episode/ and metrics/ | Episode flags, task outcomes, and diagnostics | Separate partial progress, stable completion, and safety proxies |
There is an additional training concern in the public code: a trajectory may include a return-to-home segment. The reviewed ACT loader uses meta/homing_start_sim_step and time/sim_step to determine an effective length. At the pinned revision, it falls back to the full length when the marker is absent or the computed boundary is not strictly inside the episode. A robust ingestion audit should flag missing or anomalous markers rather than assuming every sequence was trimmed. Reviewed ACT loader.
Homing frames should also be excluded consistently from normalization statistics. Before any LeRobot export, map the source timestamps, action convention, episode boundary, and feature dimensions explicitly. The LeRobot format and validation tutorial teaches these checks using synthetic data; it is not a Bench2Dex converter or a full compatibility validator.
What the reported policy scores compare
The paper evaluates four policies on 26 task–embodiment settings with 50 rollouts per setting and channel. Four channels produce 20,800 evaluation episodes in total. The design does not cross every task with every one of the 12 hands, so it cannot isolate hand quality from task difficulty. Fifty evaluation rollouts are also not 50 independent retraining runs.
Stable success requires the terminal predicate to remain true for a configured dwell time, 0.5 seconds by default. Latched stage completion records dependency-valid partial progress and is a different metric. High-speed diagnostics are motion proxies, not contact-force measurements.
| Policy | Matched scene: successes / 1,300 | Combined scene changes: successes / 1,300 |
|---|---|---|
| ACT | 383 / 1,300, 29.5% | 169 / 1,300, 13.0% |
| Diffusion Policy | 168 / 1,300, 12.9% | 50 / 1,300, 3.8% |
| π0.5 | 355 / 1,300, 27.3% | 256 / 1,300, 19.7% |
| GR00T N1.5 | 631 / 1,300, 48.5% | 258 / 1,300, 19.8% |
These are author-reported Table 2 outcomes. The combined channel independently samples both visual-context changes and geometry changes. It is not a paired, necessarily harder version of each matched episode. GR00T’s two-success lead over π0.5 under the combined shift should not be presented as a decisive general advantage. Aggregate scores also hide tasks where every policy records zero success.
Public resources and software requirements
On September 18, the official code repository and Hugging Face listings for demonstrations, assets, and checkpoints were accessible. We inspected repository revision f96a8b2, selected loader/writer code, and listing metadata; we did not download the complete data or run the simulator.
| Resource | Verified access | License and execution boundary |
|---|---|---|
| GitHub code | Public repository, pinned revision reviewed | Repository MIT license; bundled dependencies and assets need their own review |
| Demonstration files | Public, ungated HDF5 file listing | No top-level dataset card or explicit dataset license found in the reviewed listing |
| Simulation assets | Public, ungated listing | No collection-wide license confirmed; upstream asset terms may differ |
| Policy checkpoints | Public, ungated listing | Weight licensing and base-model terms require separate confirmation |
The reviewed README specifies Python 3.11, Isaac Sim 5.1.0, Isaac Lab v2.3.2, PyTorch 2.7.0, and CUDA 12.8 wheels. These are the project’s stated versions, not an installation combination independently tested by RoboSkin. Consult the official documentation and preserve a working revision; compatibility with later Isaac Lab releases is not established here.
The roughly 1,300 demonstrations are the paper’s reported collection size. We have not independently recounted or checked every hosted episode. Public download access and permission to reuse data are separate questions; the repository’s MIT license should not automatically be assigned to the dataset, assets, or model weights.
Where to go next
Compare these simulation results with physical detection and control evidence in From Tactile Sensing to Robot Action. The Bench2Dex directory record separates public file access from dataset licensing and payload validation.
Use the benchmark directory to compare evaluation questions and the dataset catalogue to compare physical and simulated sources. For a small exercise that runs without a simulator, start with Python tactile CSV processing. To contrast simulated geometric touch with learned tactile futures, read DexTouch-WM’s human-data transfer results.
Sources
Check the data behind this research
Compare the reported collection with the publicly listed files, dataset license, split documentation and dated access evidence.
Read the directory’s public availability and reproduction analysis.
Continue the topic