LiMA separates slow visual imagination from faster dexterous action refinement
LiMA amortizes a slow world-action Dreamer across faster action refinements for bimanual dexterous tasks. It improves the latency-performance trade-off on one H100, but remains vision-only and generates chunks at 325 milliseconds.
Researchers from Peking University and the Beijing Academy of Artificial Intelligence introduced LiMA on September 23, 2026. The arXiv v1 world-action model separates sparse long-horizon visual imagination from denser motion refinement, then links the two with a latent Schrödinger bridge. Across six bimanual tasks with 20 physical trials each, LiMA completed 85 of 120 trials, or 70.8%, while reporting an amortized 325-millisecond latency for each 32-step action chunk on an NVIDIA H100. Paper and version record.
Explore tactile datasets for the broader comparison.
Key takeaways
- A slow Future Dreamer generates long-horizon visual and action intent; a Motion Refiner repeatedly combines cached intent with the newest three-camera observation and robot state.
- LiMA reports 325 ms per 32-step chunk versus 600 ms for Cosmos-Policy under the same H100, batch-size and numerical-precision setup, a 45.8% reduction. This is chunk-generation latency, not latency per executed action.
- Despite language about reacting to physical changes, the evaluated system is vision-and-proprioception based. The limitations section names tactile integration as future work. Methods and limitations.
What changed
Video-generative robot policies can spend many denoising steps jointly producing future images and actions. LiMA divides that workload. Its Future Dreamer creates a sparse strategic latent from three camera views, language and robot state. The Motion Refiner remains active between Dreamer refreshes and generates a clean execution latent containing dual-arm actions, anthropomorphic-hand motion and near-term visual features.
The bridge between them is called I2SB, a latent Schrödinger Bridge Coupling. Instead of treating Dreamer output as an extra cross-attention condition, it models refinement as probabilistic transport from coarse intent to the execution trajectory. The default schedule refreshes the Dreamer once for every four Refiner updates.
The real system uses two six-degree-of-freedom UR5 arms, two 22-degree-of-freedom SharpaWave hands and three RealSense D435 cameras. Demonstrations come from VIVE wrist tracking and MetaGlove Pro finger retargeting. Each of the six tasks has 100 expert teleoperation demonstrations.
Results and comparison conditions
The evaluation includes Stack Cup, Roll T-shirt, Cook Rice, Make Sandwich, Make Coffee and Assemble Package. With 20 trials per task, LiMA recorded success rates of 90%, 75%, 80%, 70%, 60% and 50%, respectively. That sums to 85 successes in 120 trials. The paper's 78.9% progress success rate averages completion of defined subtasks and should not be confused with full-task success.
The four baselines use the same task instructions, observations, hardware interfaces and task-specific dataset where fine-tuning is required. GR00T N1.6 completed 79 of 120 trials, InternVLA-A1 80, Cosmos-Policy 72 and VPP 66. LiMA therefore leads the paper's aggregate comparison, but not every task: it ties or trails a baseline on Roll T-shirt and Assemble Package.
Latency was measured end to end from receiving multi-view observations to decoding a 32-step action chunk. LiMA amortizes one Dreamer pass across four Refiner updates, producing the reported 325 ms. GR00T N1.6 and VPP were faster at 270 ms and 225 ms, while InternVLA-A1 took 360 ms and Cosmos-Policy 600 ms. Calling LiMA real-time therefore describes its asynchronous closed-loop design; it does not mean a new chunk arrives every control timestep.
In a Cook Rice visual-generalization test, LiMA retained 70% success with novel object instances over 20 trials. Other variations changed backgrounds, illumination and clutter. These are visual out-of-distribution tests, not tactile or force robustness tests.
What this means for robotics
RoboSkin analysis: LiMA demonstrates a useful scheduling idea for robot world models. A costly predictor does not need to run at the same cadence as every corrective action. Caching long-horizon intent and refining against current observations can reduce the cost of generative foresight without removing it.
For contact-rich dexterity, the sensory boundary matters. The Refiner can react only to changes visible to its cameras or reflected in robot state. Occluded slip, distributed hand contact and incipient jamming may not be visually identifiable. Connecting the fast branch to tactile manipulation signals would test whether the asynchronous architecture handles physical contact rather than only visual change.
Limitations and availability
This is an arXiv v1 preprint whose manuscript is labeled CoRL 2026; RoboSkin.ai did not independently verify review status or reproduce the results. Experiments use one bimanual platform, six tasks, 100 demonstrations per task and an H100 GPU. Severe occlusion and low visual variance reduce performance. The latency comparison excludes model-specific compilation, quantization, operator fusion and multi-GPU optimization.
The official LiMA page provides the paper, method graphics and task videos. The November 4 review did not verify source code, training data, model weights or a project-specific implementation license through that page. Paper and media access should be distinguished from runnable assets; a page-template license would not establish the terms of separately released software, data or weights.
Separate refresh cadence, compute allocation and task meaning
Two ratios in LiMA describe different decisions. Refreshing the Dreamer once every four Refiner updates controls how often intent is renewed. The reported 3:7 slow–fast configuration allocates denoising work. Neither ratio is a measured sensor-to-actuator deadline. The 325 ms average distributes one Dreamer pass across four chunks; it does not state the worst delay when a new plan is needed. Scheduling, ablations and timing.
| Question | Evidence needed beyond the headline average |
|---|---|
| How old is the intent? | Timestamp of the observation used by the cached Dreamer output |
| When can a disturbance affect motion? | Latest Refiner observation, generated chunk, executed prefix and next query |
| Is compute actually concurrent? | Execution trace; the paper leaves further parallel/multi-GPU deployment to future work |
| What counts as task completion? | Ordered goals, physical materials and every failed or interrupted attempt |
Task names also require reading their definitions. Make Coffee uses simulated pouring motions rather than actual liquid transfer, for hardware protection. Its score does not measure dispensing volume, spillage or hot-liquid handling. Roll T-shirt failures include pinching empty space, while package assembly is affected by camera coverage and transparent parts. These examples motivate a contact-observability test; they do not demonstrate tactile capability.
The paper's six-task progress average is 78.9%. The current project abstract says 78.5% while displaying the same per-task table; this article retains the paper/table value and does not invent a reconciliation. Neither progress figure is the 85/120 complete-task result.
The learning-change checklist helps separate pretrained weights, task-specific training, cached intent and current observations when budgeting a receiving experiment.
Sources
*Source review updated November 4, 2026. Original publication dates are retained. Research results are author-reported; RoboSkin has not run these research systems or robot experiments.*
Continue the topic
