Tactile manipulation: from contact to robot action

Physical contact changes a sensor surface or taxel array. The system timestamps and preprocesses that signal, estimates a contact state or embedding, then changes grip force, pose, trajectory, or recovery mode. The action creates the next contact observation, closing the loop.

Touch should be added where it resolves a physical uncertainty. More sensor channels are not automatically useful; the controller must connect a specific tactile event to a specific action or recovery rule.

A tactile encoder compresses sensor observations. A state estimator predicts variables such as contact location, pose, force, or slip. A policy maps observations to actions. A world model predicts future contact under candidate actions. These roles may share a backbone, but the labels are not interchangeable.

Foundation-model language should be reserved for systems with broad pretraining and demonstrated downstream transfer. A strong policy on a small task suite can be valuable without being a foundation model.

Evidence becomes stronger as it moves from sensor visualization to real robot outcomes. The most useful result connects a tactile signal to an action and then shows repeatable improvement under held-out objects, disturbances, sensor replacement, or other deployment-relevant variation.

Tactile information can also close a human-in-the-loop control path. The Missing Touch study maps GelSight Mini contact images to a 32-DoF fingertip display and reports more natural, consistent motion in two 2-DoF teleoperation tasks. It measures operator trajectories, not an autonomous tactile policy, and it does not establish the same effect for multifinger robot hands or complex dexterous manipulation.

This distinction matters: feedback rendered to a human can improve how demonstrations are collected, while autonomous tactile manipulation still requires a trained model or controller and closed-loop robot evaluation.

Tactile manipulation still faces hardware diversity, limited shared datasets, inconsistent evaluation, calibration drift, contact-sequence leakage, and weak transfer across sensors and embodiments. Whole-hand and humanoid systems also add bandwidth, wiring, coverage, and safety constraints.

The practical research direction is not touch instead of vision. It is aligned vision, language, proprioception, and touch with an evaluation that shows which modality changes which physical outcome.