Tactile simulation

TouchTherm gives digital twins tactile texture and cooling dynamics

TouchTherm augments coarse collision meshes with registered tactile micro-height fields and time-varying surface temperature for multimodal simulation.

By RoboSkin.ai Editorial TeamPublished 2026-10-028 min read

Preprint · arXiv v1 · 20 reconstructed objects · tactile, thermal and VR demonstrations · code and object assets promised after acceptanceSource date: 2026-10-01Read the primary source ↗

tactile simulationthermal renderingdigital twinsGelSight microgeometry

Diagram of a scanned object carrying separate collision mesh, tactile microtexture and time-varying thermal field layers. — Original RoboSkin.ai schematic of a TouchTherm-style multimodal object asset. It is explanatory artwork, not an experimental figure.

ShanghaiTech University researchers released TouchTherm on October 1, 2026. The pipeline turns a physical object into a simulation asset with three registered layers: a coarse visual/collision mesh, local micro-height fields for optical tactile rendering and a dynamic surface-temperature field. Tests cover 20 objects, with held-out 30- and 45-second thermal predictions and synthetic-to-real tactile recognition. Paper and version record.

Key takeaways

Structured-light geometry handles object shape and collision, while photometric-stereo normals recover contact-scale relief without making the collision mesh extremely dense.
Multiview infrared videos record natural cooling after controlled heating. A physics-regularized model identifies surface diffusion and ambient relaxation, then a graph-based rollout advances the temperature field.
TouchTherm raises synthetic-to-real Top-1 object recognition from 20.0% to 34.0% in the reported setup, an absolute gain of 14.0 percentage points, but the classifier uses only 20 object classes.

From a scan to a multisensory asset

The acquisition stack combines an EinScan Pro 2X V2 structured-light scanner, smartphone image stacks under varied illumination, HIKMICRO P09 thermal cameras and a GelSight Mini for validation and per-object amplitude calibration. Normal maps are registered to the coarse mesh, transformed into a local tangent frame and integrated into micro-height residuals. At contact time, the renderer combines those residuals with coarse indentation. Full reconstruction method.

For temperature, multiple cameras observe a heated object cooling. The method fuses visible measurements onto surface points, represents spatial connectivity with a graph operator and fits two physical parameters. The network helps reconstruct unobserved regions and identify the parameters; runtime rollout retains the graph and coefficients rather than the neural field.

Results under the reported conditions

The tactile comparison uses three contacts per object across 20 objects. TouchTherm reports mean G-SSIM/HF-NCC scores of 0.0701/0.0911, versus 0.0480/0.0116 for coarse geometry and 0.0629/0.0081 for direct image-space height integration. Real-to-real repeatability is higher than real-to-sim similarity, so the result supports an improvement over those two renderers, not photorealistic equivalence to a physical sensor.

Thermal evaluation uses a separate 60-second training capture for each object's parameters and 15 overlapping test windows per object: 300 windows at each horizon. Surface-temperature MAE is 0.465 °C at 30 seconds and 0.592 °C at 45 seconds. Removing ambient relaxation increases MAE to 0.820 °C and 1.083 °C; removing surface diffusion changes the headline MAE little, which the authors interpret as ambient exchange driving global cooling while diffusion adds smaller local gains.

For recognition, a ResNet-18 trains on 90 simulated contacts per object and validates on ten. Six seeds use identical contact configurations. One hundred real images calibrate the renderer globally; a separate 20 real images per object are held out for evaluation, and no real image trains the classifier. Top-1/Top-3/Macro-F1 improve from 20.0/35.0/13.4% with coarse geometry to 34.0/63.0/29.3% with TouchTherm. A separate VR demo with ten participants reports perception and comfort ratings, but it is a small usability demonstration rather than a robotics task benchmark.

RoboSkin analysis

TouchTherm moves tactile simulation from “object mesh plus sensor model” toward an object-side contact asset. That separation matters: the geometry needed for stable collision can stay coarse while the vision-based tactile sensor queries finer surface relief. Temperature adds another field that could support material-aware teleoperation or Physical AI touch data, although no temperature-conditioned robot policy is evaluated.

The half-day-per-object estimate for each acquisition modality is also an integration warning. Scaling beyond 20 objects requires automation of manual illumination, 2D–3D correspondences, heat excitation and calibration. The paper demonstrates a pipeline, not a ready-made large catalog.

Limitations and availability

TouchTherm is an arXiv v1 preprint under double-anonymous review, and RoboSkin.ai has not reproduced it. Manual registration and one-time per-object GelSight amplitude calibration remain in the workflow. Recognition uses one optical tactile sensor style; thermal feedback is demonstrated in VR rather than on a robot controller.

The official project exposes videos and method descriptions. It states that code, reconstructed object assets and the simulation pipeline will be released upon acceptance. No downloadable archive, repository, dataset license or software license was verified on October 2, 2026. The paper itself uses arXiv's perpetual non-exclusive license, which is not an implementation license.