Whole-hand force regulation tracks contact without tactile sensors

Researchers at Seoul National University, Massachusetts Institute of Technology and Yonsei University released a real-time whole-hand force controller on September 24, 2026. It estimates contact across fingertips, finger sides, dorsal surfaces and the palm from a tracked object model and joint angles, then reallocates desired forces as those contacts change. The method explicitly does not use tactile sensing at the contacts. Paper and version record.

Explore tactile sensing for the broader comparison.

Key takeaways

• About 3,000 sampled hand-surface points are queried against the object's signed distance field every cycle. That geometric contact search takes under 1 ms on the reported GPU.
• In gravity-free simulation, whole-hand regulation reaches 88.6% success after a 90-degree perturbation versus 72.4% when contact candidates are restricted to fingertips.
• Hardware evidence consists of three uncut runs totaling about 200 seconds, 78 human-applied perturbations and eight object losses; the system autonomously recovers six losses. Official hardware summary.

What changed

Grasp planners usually choose contacts once, then execute forces or impedance targets tied to that initial geometry. That becomes brittle when an object rolls, slides or gains new contacts against another part of the hand.

This controller treats contact as a live geometry query. The object mesh is converted to a signed distance field, and the measured hand configuration places sampled points from every link into that field. Points within a threshold become contact candidates. For each current set, a quadratic program allocates force under friction pyramids, minimum normal force, joint-torque limits and an actuation-consistency constraint.

The last constraint is important. An optimizer can request forces that satisfy object-level equations but cannot be generated by controlled finger torques. The method suppresses force components in the null space of the actuated contact Jacobian. It remains a conservative allocation rule, not proof that compliant physical contacts realize every desired vector.

Reactive reaching handles acquisition. It follows a tracked object until the thumb and two fingers make contact, hands control to force regulation, and returns to reaching after a detected loss. This creates an approach-hold-regrasp loop rather than evaluating a stored grasp alone.

Simulation and hardware evidence

Simulation uses MuJoCo with gravity disabled, 79 YCB objects and ten power grasps per object. A six-direction load equal to 10% of object weight tests each initial grasp. For dynamic perturbations, five representative objects and 50 grasps per method are rotated up to 90 degrees; non-adaptive baselines allocate force only at initialization. Whole-hand online regulation beats fixed feedforward torque, fixed impedance and fingertip-only execution at every angle. Simulation protocol.

Under constant synthetic pose errors up to 30 degrees and 5 mm, the feedforward baseline falls from 79% to 54%, while adaptive regulation stays between 87% and 93%. That test does not include time-varying noise, latency or complete tracking loss.

There is a measurable speed-coverage trade-off. Allowing one candidate per link gives a 227 Hz force solve and 90.9% success; five candidates peaks at 93.2%; seven slows to 66 Hz and falls to 91.9%. The selected five-contact setting has a mean solve time whose reciprocal is about 84 Hz. That computation benchmark is not the physical controller update rate: the hardware contact search and solve run once per 30 Hz tracked pose, paired with joint state at the image timestamp.

Hardware uses a 7-DoF Flexiv Rizon 4 arm, a 20-DoF Robotis 5F hand, one RealSense D455, an Intel i9-13900K and an RTX 4090. The hand's fingertip tactile sensors are present but unused. A Pringles can and two boxes are pushed or taken during the three runs. Six of eight losses are recovered autonomously; the other two leave the arm workspace and require a person to reposition the object and restart. These demonstrations show autonomous transitions, but three runs are not a comparative hardware benchmark.

What this means for robot skin

RoboSkin analysis: this work is a useful counterexample to the claim that every contact-control problem requires dense skin. If the object model, pose tracker and hand kinematics are reliable, geometry can infer likely contacts over surfaces that have no taxels at all.

That does not eliminate the case for [robot skin](/). It relocates it. A suitably placed and calibrated tactile layer could supply local evidence of contact or slip when geometry is uncertain; which quantities it measures depends on the sensor and estimator. Geometry offers broad coverage; touch offers physical confirmation. A hybrid system could use this controller as a contact prior and tactile residuals to correct model error.

Limitations and availability

The method requires an object model from CAD or a short RGB-D scan and a maintained six-degree-of-freedom pose. Heavy occlusion breaks contact estimation. Thin objects are especially sensitive because millimeter-scale pose error can flip a sampled point to the far side and reverse the estimated normal.

Force allocation is quasi-static. It does not explicitly compensate gravity, inertia or external wrenches, even though friction is expected to resist unmodeled loads. Simulation disables gravity, and hardware disturbances are manual rather than standardized.

The project supplies the paper, videos and detailed results. Its code link is labeled “Soon”; no repository, data package or software license was verified on September 26, and the November 4 project-page check still shows that pending code control. The project-page content is CC BY-SA 4.0, which licenses that page, not unreleased controller code. RoboSkin.ai has not reproduced the system.

Check stale state before trusting the force command

The rendered force arrows are commands, not measured contact forces. Contact locations come from geometry, so a plausible visualization does not independently verify the physical force distribution. The paper also retains the last feasible command when the optimization fails, starting from zero before a feasible solution exists. Controller and hardware timing.

For an integration review, ask how old the object pose may be, which joint sample it is paired with, and how a failed solve or lost tracker is signaled. Record any human repositioning separately from automatic regrasp. These are proposed acceptance questions; the paper does not establish a validated stale-data watchdog for a new platform.

Use the tactile manipulation interface comparison to distinguish a geometry-derived contact hypothesis from a measured force input, a desired force and the controller that executes it.

Sources and related resources

• arXiv v1, submitted September 24, 2026
• Full controller and evaluation
• Official project and uncut hardware runs
• RoboSkin dexterous robot hands guide

*Evidence review updated November 4, 2026. The original paper and article dates are retained; source inspection is not a RoboSkin.ai reproduction.*