DexTouch-WM learns robot contact dynamics from human touch

Evidence review — September 17, 2026 preprint; sources checked September 18, 2026

DexTouch-WM is an action-conditioned world model that predicts future visual observations and touch for dexterous robot manipulation. The paper by Qin and colleagues studies whether tactile demonstrations collected from people can improve a model trained with limited robot data. Its most useful finding has two parts: human pretraining improves the reported contact predictions, while training policies on generated trajectories produces mixed real-robot outcomes.

Those are different tests. A model can become better at predicting contact or ranking policies without generating training data that reliably replaces real demonstrations. This distinction matters when choosing between collecting more robot data, adding human demonstrations, and using synthetic rollouts.

What the model observes and predicts

The model conditions on an initial RGB observation, left- and right-hand tactile maps, a language instruction, and a future action-pose sequence. It generates future RGB and tactile observations. A visual expert based on Wan2.2-TI2V-5B works with a lightweight tactile expert; the future actions are conditioning inputs, rather than evidence that the world model independently chooses safe robot commands. Method and experiments.

Human collection uses Moxian piezoresistive tactile gloves, Manus MetaGloves, Vive wrist trackers, and wrist/head cameras. The raw tactile gloves have 360 taxels per hand and sample at 60 Hz. The training representation retains 320 taxels: five 4 × 4 fingertip pads and one 15 × 16 palm pad. The robot uses the same retained tactile layout on a Tianji arm and a 20-DoF Wuji hand configuration.

The 60 Hz figure describes tactile acquisition. It is not a measured model inference rate or an end-to-end control frequency. Likewise, matching the 320-taxel layout does not remove the need for spatial alignment, clock synchronization, and motion retargeting. Human and robot motions are mapped into a common 67-dimensional pose representation. Transfer to an arbitrary hand, glove, or camera-only human video is not established by this experiment.

For the practical collection side, see robot teleoperation and demonstration quality. Preserve the original clocks, sensor layout, validity flags, and calibration record before converting observations into a common model representation.

Contact prediction improves with large-scale human pretraining

The pretraining comparison holds robot data fixed at five hours from six tasks. The largest human-data condition adds 100 hours from 50 human tasks; the human and robot pretraining task sets are disjoint. Four downstream tasks form a separate evaluation stage.

The following values come from Table I. They compare robot-only pretraining with the condition that adds 100 hours of human data. They are author-reported prediction metrics, not RoboSkin measurements.

The end-point comparison supports improved visual and contact prediction in this setup. It does not show that every increment of human data improves every metric: the paper’s smaller-data conditions include regressions. Contact overlap and F1 also do not measure calibrated force error. Readers should retain the source’s signal definitions rather than assigning newton or pressure units to a normalized tactile value.

Policy evaluation and policy training answer different questions

The downstream study compares a world model adapted with 200 robot trajectories, WM-Robot, with WM-Mix, adapted with 100 robot and 100 human trajectories. These adaptation demonstrations are separate from the human pretraining collection. Policies are evaluated using ten matched rollouts per policy, task, and environment, with five human raters scoring each rollout.

The reported task scores are normalized to the interval from zero to one. They are not binary success rates. Five raters do not turn ten robot rollouts into 50 independent robot trials. The imagined-rollout scoring also penalizes obvious physical inconsistency.

For world-model-based policy evaluation, the reported mean Pearson correlation with real scores increases from 0.646 for WM-Robot to 0.844 for WM-Mix. This is a limited comparison involving three policies per task, and a ranking error remains on Stand Bottle. Better agreement in this evaluation does not establish a universal replacement for real-robot testing.

Generated training data still has a downstream cost

Table IV tests policies on the real robot after three training-data choices. Each row below reports the mean normalized task score across four tasks; larger is better. It does not report a success percentage.

The mixed human/robot world model does not consistently produce the best policy-training data. For π0.5 and X-VLA, its generated-data condition is below both alternatives in the four-task mean. X-VLA also receives a zero score on Stand Bottle in that condition. These failures deserve to remain visible alongside the improved prediction metrics.

The generated trajectories retain real initial observations and recorded robot actions while replacing future RGB and tactile observations with predictions. This is not a pipeline that creates complete demonstrations without any robot data. The source separates the held-out trajectories used for generation from those used to adapt the world model.

What to carry into a data workflow

• Keep physical observations, human observations, and generated observations distinguishable at episode and field level. Record the generating model and its revision for synthetic data.
• Split complete trajectories before adaptation or generation, and record which initial observations and actions remain real.
• Evaluate visual quality, contact prediction, policy ranking, and final robot task performance separately. A gain in one is not evidence for all four.
• Store raw tactile units and calibration provenance. Use the tactile calibration workflow to separate image or signal baselines from force estimation.
• Validate episode boundaries, timestamps, and array dimensions before training. The LeRobot data-format guide explains these checks; its teaching validator does not convert or certify DexTouch-WM data.

For the broader architecture context, compare the five-system visuo-tactile world-model guide. For simulated contact observations with an explicit benchmark protocol, read the Bench2Dex resource review.

Publication and access boundary

The arXiv record identifies this work as accepted to the IROS 2026 RoBoWoMo workshop as a lightning talk. That is distinct from acceptance as an IROS main-conference paper. This review uses the September 17 v1 preprint.

No official code, model-weight, or dataset download was found in the paper and arXiv record reviewed on September 18. This is an access finding for those sources, not a claim that no release can exist elsewhere or appear later. RoboSkin reviewed the methods and result tables; we did not train the model, collect glove data, or reproduce the hardware experiments.

Sources

• DexTouch-WM arXiv record and publication note
• DexTouch-WM v1: method, Tables I–IV, and experiment definitions