Continual 6-DoF grasping learns from outcomes without weight updates
A memory-based grasp pipeline turns successes, failures and optional demonstrations into immediate proposal and score changes instead of repeatedly fine-tuning a network.
ETH Zurich researchers released a continual-learning system for single-view 6-DoF grasp synthesis on October 1, 2026. The CoRL 2026 paper keeps its neural encoder fixed after deployment. Successful and failed grasps update a local memory-based score, while optional user demonstrations contribute new candidates through geometric registration. The evaluation includes more than 1,500 physical grasp attempts. Paper and version record.
Key takeaways
- A 32-dimensional geometric embedding lets nearby successes and failures update a Beta posterior for each proposed grasp without backpropagation.
- A second memory registers demonstrated grasps onto similar object regions, adding candidates that fixed contact-normal or top-down heuristics miss.
- After at most 50 adaptation attempts per category, the system exceeds 90% grasp success in five of six physical object groups, but reaches only 68.3% on pliers.
How the two memories change grasping
The base system follows a sample-and-score pipeline. It builds grasp candidates from a partial point cloud, encodes a local patch around each candidate and ranks the candidates before collision checking. Deployment outcomes are automatically labeled from gripper width and written to scoring memory. At the next scene, nearby memory entries adjust the estimated success probability rather than changing the encoder weights.
Demonstration recall addresses a different failure. If the geometric sampler never proposes the needed pose, better scoring cannot recover it. The user can provide a grasp, which is stored with local shape descriptors and later transferred to geometrically similar regions. This division is important: outcome memory refines selection among available candidates; demonstration memory expands the candidate set. Method and evaluation protocol.
Simulation and physical evidence
Simulation uses 443 unseen objects across ten categories, with 2,500 attempts per category and method. Before online adaptation, the proposed base averages 94.6%, compared with 92.9% for EdgeGraspNet. Category-specific full adaptation raises the average to 98.1%; scoring-only reaches 97.1%, while recall-only reaches 85.6%. That ablation suggests the non-parametric scorer drives most average improvement, while recall remains useful for missing proposal modes.
The physical setup uses a Franka Panda, parallel-jaw gripper and RealSense D435i. Fifty-four objects form one 13-object control set and five challenge groups: mugs and bowls, kitchen tools, pliers, screwdrivers and toys. Each category has 20 fixed evaluation scenes per method. Because scenes contain multiple objects, the number of attempts varies with how quickly a method clears a scene; “more than 1,500” is therefore the supported total, not a uniform trials-per-cell claim.
After at most 50 adaptation attempts and a demonstration after each failure, success moves from 80.0% to 93.2% on controls, 92.7% to 100% on mugs and bowls, 77.4% to 91.3% on kitchen tools, 62.5% to 68.3% on pliers, 85.7% to 96.0% on screwdrivers and 83.6% to 94.4% on toys. A merged memory evaluated on 20 mixed scenes reaches 89.6% grasp success and clears all scenes, versus 72.4% success and 85.0% scene clearance for the base model.
RoboSkin analysis
The most useful result is not the five “over 90%” categories in isolation. Pliers expose what local geometry cannot resolve: one instance needs a strategy that conflicts with the others, so pooled evidence favors the majority and repeatedly fails the exception. This is a practical warning for robot grasp learning: a memory can adapt immediately, but only if its similarity space separates the states that demand different contact strategies.
The work also separates contact proposal from contact evaluation, a useful design pattern for robot hands. Failures caused by low friction or uneven mass can update scores; failures caused by missing depth geometry may require a demonstrated pose or another sensor. Teams working on slip-aware robot hands should not treat a binary gripper-width label as a substitute for direct slip or force observations.
Limitations and availability
The experiments use one parallel-jaw platform and geometry-only observations. Reflective, transparent and metallic surfaces can corrupt depth, and memory cannot recover geometry absent from the input. Recall cost grows with demonstrations; the reported sequential simulation ends at 1.35 MB, but larger deployments would need filtering. The study focuses on difficult object categories rather than systematic variation in sensor noise or contact dynamics.
The official project publishes videos and identifies the work as CoRL 2026, but its code link is labeled “Coming Soon.” No repository, trained model, grasp-memory archive, physical dataset or implementation license was verified. RoboSkin.ai did not run the system.
Sources and related resources
Continue the topic
