Res-HIL learns corrections around a frozen manipulation policy
Siemens and Technical University of Munich researchers use interventions both as corrective targets and reward shaping. The real-robot study is strong on task coverage, but still depends on one operator and unreleased code.
Researchers at Siemens and the Technical University of Munich introduced Res-HIL on September 24, 2026, for sample-efficient adaptation of contact-rich robot skills. Instead of replacing a behavior-cloning policy during online learning, the method freezes it and trains a residual correction. Human interventions supervise that correction and penalize the autonomous actions that led toward failure. Paper and version record.
Key takeaways
- Every online method starts from 20 demonstrations; frozen ACT and unguided ResFiT reference policies also receive 100-demonstration variants.
- After ten minutes, Res-HIL reports 100%, 64%, 50%, 66% and 92% success across five task conditions. HIL-SERL reaches 90%, 30%, 10%, 0% and 0% under the same table.
- Final evaluations use 50 fixed randomized configurations, three training repetitions and one intervention operator. That is substantial real-robot evidence, but not independent validation across people or platforms.
What changed
Human-in-the-loop reinforcement learning can adapt a robot online, but changing the entire policy risks discarding useful behavior already learned from demonstrations. Residual learning preserves that base and learns only an additive correction. Res-HIL combines the two ideas while addressing an unstable start: the residual output layer is initialized to zero, so the deployed action initially equals the base action.
During training, the operator takes over when failure, damage or an unrecoverable deviation seems likely, and releases control after returning the robot to a recoverable state. The human action becomes a target residual relative to the frozen base. Res-HIL also applies decaying penalties to autonomous steps immediately before the first two intervention segments, turning each correction into both a supervised example and a reward-shaping signal.
Two replay buffers keep online experience separate from initial demonstrations and intervention transitions. Training batches draw equally from both. A TD3 actor-critic objective learns the residual at 7 Hz, behavior cloning matches human corrections, and a magnitude penalty discourages unnecessarily large changes.
Results under the reported conditions
The five conditions are easy and hard peg insertion, vent-lid insertion, and cable manipulation with two or three cameras. The base ACT policy uses 20 demonstrations. Online budgets are 20 minutes for easy peg insertion and 45 minutes for the other tasks. Each evaluation uses the same 50 predefined randomized starts across methods, and every online experiment is repeated three times. Protocol and full tables.
After ten minutes, Res-HIL is already the strongest reported method on every condition. Final success is 100%, 96%, 100%, 88% and 100%. The 20-demonstration ACT base reaches 36%, 28%, 44%, 46% and 70%; ACT with 100 demonstrations reaches 84%, 60%, 80%, 80% and 92%.
HIL-SERL eventually reaches 100%, 94% and 84% on the three insertion tasks but 0% on both cable conditions. Its vent score uses a stricter edge-first criterion; it would be 100% if pushing the lid directly through were counted. Res-HIL averages 5.46 seconds across successful task episodes, compared with 5.83 seconds for 100-demonstration ACT and 7.73 seconds for 20-demonstration ACT. HIL-SERL is faster on the insertion tasks it solves, so Res-HIL's strength is coverage rather than the fastest motion everywhere.
The ablation is narrower than the main evaluation. On easy peg insertion, removing zero initialization doubles convergence time from eight to 16 minutes. Removing intervention-aware reward shaping prevents convergence within 30 minutes and finishes at 92%. The paper says direct residual supervision is critical, but these component tests do not establish identical effects on the long-horizon cable tasks.
What this means for robotics
RoboSkin analysis: Res-HIL is relevant to contact-rich manipulation even though its policy observes cameras and proprioception rather than tactile arrays. It treats rare contact failures as correction opportunities instead of asking a new policy to rediscover the entire behavior online.
For tactile systems, the same architecture could preserve a visual-proprioceptive base while a residual learns corrections from touch. But that extension remains an inference, not a result in this paper. The study does not test tactile input, force feedback or whether interventions transfer between operators.
The comparison with 100 demonstrations also needs context. The authors estimate collecting 100 demonstrations takes about 30 minutes for peg insertion, 45 minutes for vent insertion and 60 minutes for cable manipulation. Res-HIL reduces initial demonstrations, but it adds online robot time plus a human who monitors and occasionally takes control. Sample efficiency is therefore a trade between offline demonstration and supervised online interaction, not zero human effort.
Limitations and availability
Res-HIL is an arXiv v1 preprint on one robot setup. All interventions come from the same operator under a stated qualitative criterion. The paper reports that intervention ratios fall during training, but operator-to-operator variation, delayed takeover and safety outside the tested fixtures remain unknown.
The paper acknowledges that concurrent FTC and HiL-ResRL methods were not reproduced because they appeared shortly before submission. Comparisons therefore cover the listed baselines rather than every current human-guided residual approach.
The official project page provides task videos and a method overview. No public code repository, checkpoint, training data or software license was verified on September 26. The paper itself is available under CC BY 4.0; that license does not make absent implementation artifacts available. RoboSkin.ai has not run the training system.
Sources and related resources
Continue the topic
