Body-grounded replanning lets joint load change the robot strategy
Microsoft Research Asia, Waseda, Chiba and NII researchers move proprioceptive evidence into high-level replanning. Hardware success reaches 96.3%, while LLM pause time is excluded.

Researchers at Microsoft Research Asia–Tokyo, Waseda University, Chiba University and Japan's National Institute of Informatics released Body-Grounded Replanning on September 24, 2026. The framework turns joint position, actuator effort, contact and recent execution history into high-level evidence: when the current motion becomes physically unsuitable, GPT-4o-mini selects another strategy while the task objective and low-level controller remain unchanged. Paper and version record.
Explore tactile datasets for the broader comparison.
Key takeaways
- In ten real-robot reaching conditions, body-grounded replanning and an environment-only search baseline both reach 96.3% success; the main advantage is faster restricted completion, 24.57 seconds versus 28.18 seconds.
- A rule-based controller using the same body-state events reaches 66.3%, showing that detecting strain or contact is not equivalent to choosing the right alternative.
- The three contact-rich tasks—button pressing, box-lid opening and lint-roller extraction—are qualitative demonstrations without repeated success rates.
What changed
Most manipulation planners ask whether a trajectory is geometrically feasible. This work adds a second question: is the strategy physically suitable for the robot's current body state? A path may still reach the target while producing high joint effort, approaching a joint limit or encountering contact earlier than expected.
The monitor records per-joint position, velocity and effort plus binary contact at 20 Hz. Current measurements and a learned short-horizon prediction trigger events such as high effort, limited mobility, no progress or early contact. GPT-4o-mini, run at temperature zero, receives the current strategy, joint-level state, recent execution statistics, event summary, history and a manually defined menu of alternative strategies. Its structured choice changes the high-level motion while leaving the low-level controller in place.
Results under the reported conditions
The controlled simulation uses a Franka Panda and generates 100 paired episodes for each of ten load or asymmetric-mobility conditions. The body-grounded method maintains 90% to 99% success and records the lowest mean joint-torque norm in every condition. Across all ten simulated conditions, overall success is 96.7%, compared with 88.5% for the rule-based body-state baseline. Full results.
The hardware study uses a CRANE-X7 with Dynamixel actuators. Environment-only search, rules and body-grounded replanning each receive eight trials per condition under a 45-second execution budget. The body-grounded and environment-only methods both average 96.3% success; rules average 66.3%. Body grounding has the lowest restricted completion time in nine of ten conditions and the lowest overall mean at 24.57 seconds. Its average actuator-effort norm is also lowest—2.18 versus 2.28 and 2.33—but it wins that metric in only five of ten individual conditions.
Restricted completion time assigns the full 45 seconds to failures and excludes LLM inference pauses. That makes the metric useful for combining reliability and active robot time, but it is not end-to-end wall-clock latency. The paper does not publish a separate distribution for network or model response time.
The authors also show one execution each for obstacle-avoiding button pressing, box-lid opening and lint-roller extraction. Contact or impeded motion triggers a strategy change and task completion in the shown sequences. These examples demonstrate interface reuse, not a quantified manipulation benchmark.
What this means for robotics
RoboSkin analysis: body-grounded replanning moves proprioceptive and contact signals above the servo layer. Instead of only correcting torque locally, the robot can decide that the entire approach direction, posture or action order is wrong for its current physical condition.
The comparison with environment-only search is equally important. Both succeed at the same rate on hardware, so the evidence supports efficiency and physical suitability more strongly than raw completion. A production system would also need bounded fallback behavior when the language model is slow, unavailable or selects an unsafe strategy.
Limitations and availability
This is an arXiv v1 preprint. The high-level strategy set and event thresholds are manually defined, tasks are short-horizon, and the strongest repeated evaluation is controlled reaching rather than the three contact-rich demonstrations. Hardware effort uses raw Dynamixel readings, not calibrated joint torque, so values are only compared within that platform.
The paper links an author-provided qualitative video, and Figure 3 supplies prompt text, but no executable integration package, public code repository, logs, dataset or software license was verified. A printed prompt is not a reproduced deployment. GPT-4o-mini is an external hosted model, and the evaluation excludes its inference pauses from active execution time. RoboSkin.ai has not reproduced the system.
Record the event, the strategy and both elapsed-time clocks
The learned predictor is a short-horizon joint-state estimator that informs one effort-near-limit trigger. It is not a world model that evaluates the physical consequences of arbitrary candidate actions. The language model chooses from nine predefined strategies, and only the structured strategy field controls the robot; explanatory text is not executed as a motor command. Monitor and replanning method.
At 20 Hz, a fifteen-step monitoring window covers 0.75 seconds, while ten-step persistence and cooldown rules each span 0.5 seconds. These are different from language-model response time. The reported completion metric excludes those inference pauses, so it cannot establish end-to-end cycle time.
| Record | Why it matters |
|---|---|
| Body trace and event | Distinguishes raw effort or contact evidence from an inferred event label |
| Strategy choice and allowed menu | Shows whether the response was available under the same constraints |
| Motion-only time | Reproduces the paper's active execution metric and failure-horizon penalty |
| Wall-clock time and pauses | Exposes model/network delay, waiting and actual task duration |
Hardware effort is a raw Dynamixel quantity, not calibrated joint torque, contact force or energy consumption. The rule baseline also lacks the continuous body context supplied to the language model. A comparison therefore tests the complete information-and-selection design, not whether language models are uniquely necessary for adaptation.
For a proposed receiving test, log rejected choices and unavailable responses as well as successful strategy changes, with a separately defined fallback. That is evaluation guidance, not a capability implemented or tested here. The prediction/control comparison places this method beside world-action systems without relabeling it as one.
Sources and related resources
- arXiv v1 record, submitted September 24, 2026
- Full framework, tables and limitations
- Author-provided experiment video
- RoboSkin Physical AI overview
*Source review updated November 4, 2026. Original publication dates are retained. Research results are author-reported; RoboSkin has not run these models or robot experiments.*


