<- Back to news

Continual 6-DoF grasping learns from outcomes without weight updates

A memory-based grasp pipeline turns successes, failures and optional demonstrations into immediate proposal and score changes instead of repeatedly fine-tuning a network.

CoRL 2026 paper · arXiv v1 · public benchmark/model documentation and MIT code licenses reviewed · no payload or execution verifiedSource date: Read the primary source ↗
continual grasp learning6-DoF grasp synthesisoutcome memorydemonstration recall
Diagram showing grasp attempts feeding outcome memory and demonstration memory before the robot selects a new six-degree-of-freedom grasp.
Original RoboSkin.ai schematic of outcome-scored and demonstration-recalled grasp adaptation. It is explanatory artwork, not an experimental figure.

ETH Zurich researchers released a continual-learning system for single-view 6-DoF grasp synthesis on October 1, 2026. The CoRL 2026 paper keeps its neural encoder fixed after deployment. Successful and failed grasps update a local memory-based score, while optional user demonstrations contribute new candidates through geometric registration. The evaluation includes more than 1,500 physical grasp attempts. Paper and version record.

Key takeaways

  • A 32-dimensional geometric embedding lets nearby successes and failures update a Beta posterior for each proposed grasp without backpropagation.
  • A second memory registers demonstrated grasps onto similar object regions, adding candidates that fixed contact-normal or top-down heuristics miss.
  • After at most 50 adaptation attempts per category, the system exceeds 90% grasp success in five of six physical object groups, but reaches only 68.3% on pliers.

How the two memories change grasping

The base system follows a sample-and-score pipeline. It builds grasp candidates from a partial point cloud, encodes a local patch around each candidate and ranks the candidates before collision checking. Deployment outcomes are automatically labeled from gripper width and written to scoring memory. At the next scene, nearby memory entries adjust the estimated success probability rather than changing the encoder weights.

Demonstration recall addresses a different failure. If the geometric sampler never proposes the needed pose, better scoring cannot recover it. The user can provide a grasp, which is stored with local shape descriptors and later transferred to geometrically similar regions. This division is important: outcome memory refines selection among available candidates; demonstration memory expands the candidate set. Method and evaluation protocol.

Simulation and physical evidence

The main category-wise simulation comparison uses 443 unseen objects across ten categories, with 2,500 attempts per category for the base and full adaptation methods. Before online adaptation, the proposed base averages 94.6%, compared with 92.9% for EdgeGraspNet. Category-specific full adaptation raises the average to 98.1%; scoring-only reaches 97.1%, while recall-only reaches 85.6%. That ablation suggests the non-parametric scorer drives most average improvement, while recall remains useful for missing proposal modes.

The physical setup uses a Franka Panda, parallel-jaw gripper and RealSense D435i. Fifty-four objects form one 13-object control set and five challenge groups: mugs and bowls, kitchen tools, pliers, screwdrivers and toys. Each category has 20 fixed evaluation scenes per method. Because scenes contain multiple objects, the number of attempts varies with how quickly a method clears a scene; “more than 1,500” is therefore the supported total, not a uniform trials-per-cell claim.

After at most 50 adaptation attempts and a demonstration after each failure, success moves from 80.0% to 93.2% on controls, 92.7% to 100% on mugs and bowls, 77.4% to 91.3% on kitchen tools, 62.5% to 68.3% on pliers, 85.7% to 96.0% on screwdrivers and 83.6% to 94.4% on toys. A merged memory evaluated on 20 mixed scenes reaches 89.6% grasp success and clears all scenes, versus 72.4% success and 85.0% scene clearance for the base model.

RoboSkin analysis

The most useful result is not the five “over 90%” categories in isolation. Pliers expose what local geometry cannot resolve: one instance needs a strategy that conflicts with the others, so pooled evidence favors the majority and repeatedly fails the exception. This is a practical warning for robot grasp learning: a memory can adapt immediately, but only if its similarity space separates the states that demand different contact strategies.

The work also separates contact proposal from contact evaluation, a useful design pattern for robot hands. Failures caused by low friction or uneven mass can update scores; failures caused by missing depth geometry may require a demonstrated pose or another sensor. Teams working on slip-aware robot hands should not treat a binary gripper-width label as a substitute for direct slip or force observations.

Limitations and availability

The experiments use one parallel-jaw platform and geometry-only observations. Reflective, transparent and metallic surfaces can corrupt depth, and memory cannot recover geometry absent from the input. Recall workload grows with demonstrations under a fixed time budget; the reported sequential simulation memory ends at 1.35 MB, which is not total system memory. Larger deployments would need to measure the resulting recall tradeoff. The study focuses on difficult object categories rather than systematic variation in sensor noise or contact dynamics.

The November 4 review found that the official project now links public benchmark code and model code, both carrying MIT software licenses. Their READMEs describe simulation and Panda integration, and link base/adapted modules and synthetic training data. Those external payloads were not downloaded or verified here; third-party object assets and submodules require their own checks. Source documentation is available, but RoboSkin has not executed the code or reproduced the experiments.

Version the memory as carefully as the neural checkpoint

A frozen encoder does not make the deployed behavior fixed. Outcome memory changes candidate scores, while demonstration memory can introduce a previously absent grasp. The weighted Beta update uses similarity-weighted evidence and source-dependent weighting; its score should not be treated as a calibrated probability of safe completion. A demonstrated pose can also receive a positive label without an executed success. Scoring, recall and evaluation appendices.

Evidence layerWhat a receiving team should retain
Observation and labelPoint cloud, gripper-width outcome, label origin and any independently observed slip or failed lift
Adaptation stateEncoder revision, offline memory, accumulated outcomes and demonstrated poses
Evaluation boundaryWhether feedback was permitted, which object instances were held out and when memory was frozen
Search budgetCandidate source, recall budget, missed registrations and time spent per demonstration

The sequential simulation visits ten categories in one order. The mixed-scene hardware result instead merges independently adapted category memories. Both are useful, but the latter does not test uninterrupted lifelong adaptation on a physical robot. Report those protocols separately when evaluating forgetting or conflicting examples.

The implementation also trades recall coverage for bounded time: a fixed recall budget is divided among stored demonstrations. Increasing memory can therefore reduce search time per item even without increasing the total runtime. Record missed proposals as well as failed executions; a higher score cannot rescue a grasp that was never generated.

The public model README specifies a Panda-specific grasp-center frame, rather than the robot flange, and separates base from adapted modules. A different gripper requires more than changing a width value. The robot-learning adaptation checklist keeps memory changes, parameter updates and runtime conditioning distinct. When direct contact observations are needed, compare the measured signals and access boundaries in the tactile dataset directory.

*Source review updated November 4, 2026. Original publication dates are retained. Research results are author-reported; RoboSkin has not run these research systems or robot experiments.*

Continue the topic

Tactile robot learningA tactile reflex becomes the teacher for fragile graspingHumanoid benchmarksHumanoidToolBench separates tool choice from task completionHaptic teleoperationDITTO-X makes a robot hand align the operator before takeover