<- Back to news

InterEvolve searches reward programs instead of retraining a humanoid

InterEvolve uses an LLM, numerical tuning and parallel simulation to revise staged reward programs while keeping the Unitree G1 control policy fixed.

Preprint · arXiv v1 · simulation search over eight task families · physical G1 demonstrations · project videos public · no code, data or software license verifiedSource date: Read the primary source ↗
humanoid loco-manipulationreward program searchtest-time adaptationwhole-body control
Diagram showing an LLM editing staged reward code, simulation verification and a frozen humanoid controller executing the selected program.
Original RoboSkin.ai schematic of InterEvolve's reward-program search loop. It is explanatory artwork, not a robot experiment frame.

University of Illinois Urbana-Champaign researchers released InterEvolve on October 1, 2026. The system adapts a frozen humanoid controller to new contact-rich tasks by changing the reward program that queries it, rather than updating policy weights. An LLM edits program stages, CMA-ES tunes numerical constants, and parallel simulation verifies proposals. The reported macro-average rises from 34.6% for a tuned initial agent program to 86.5% after evolution. Paper and version record.

Explore tactile datasets for the broader comparison.

Key takeaways

  • An object-aware forward-backward behavioral foundation model turns rewards into latent prompts for one fixed Unitree G1 controller.
  • Full InterEvolve reports 86.5% macro-average simulation success across eight task families. The reported 2.1 GPU-hours and 0.23 million LLM tokens cover a complete evolution run for one family, not the combined eight-family experiment.
  • Real-robot demonstrations use an RGB-D camera on the robot, but perception, state estimation and policy inference run on an off-board GPU workstation.

Evolve the interface, not the weights

The motor model starts with a frozen body prior and adds object-aware residual networks trained on retargeted OMOMO and GRAB interactions. After holding out 50 clips for each of four box-like objects, 3,866 clips remain for training. At test time, a reward is projected against a state bank into a latent prompt that elicits behavior from the fixed policy.

A reward program can contain multiple stages, completion conditions, weights and thresholds. DeepSeek-V4-Flash revises the program structure from simulator feedback, while CMA-ES calibrates constants. Each valid proposal receives 192 tuning rollouts; top candidates receive a separate 192-rollout confirmation evaluation. Verified programs become a text skill library that later tasks can retrieve. Method, search budget and evaluation details.

This is adaptation through task specification. It can expose behavior already present in the controller, but it does not manufacture a missing motor primitive. The same distinction appears in contact-rich Physical AI: better objectives can coordinate pushing, lifting and carrying only if the underlying control distribution contains usable versions of those motions.

What improved—and what did not

Across eight large-box task families, the initial agent program with numerical tuning reaches 34.6% success. The full method reaches 86.5%. Removing CMA-ES reduces that result to 51.6%; forcing one stage reduces it to 44.7%; removing targeted edits, multi-scenario testing or scene context produces 68.9%, 68.4% and 78.7%.

The progression is not monotonically successful. Full success moves from 32.2% initially to 75.0%, 76.8%, 73.6% and finally 86.5% over four rounds. The smoother “earned criteria” metric rises every round, but a change that fixes one criterion can temporarily break another when all conditions must hold in the same rollout.

Skill-library reuse is tested on three composite tasks with ten episodes each. The full library completes 8/10 relocate, 4/10 stack and 5/10 carry-place-kick episodes. With no library those counts are 0/10, 0/10 and 1/10. These cells are small and task-specific, but they show that verified task descriptions can be reusable artifacts.

RoboSkin analysis

The ablations make stage structure and calibration inseparable. A high-level program such as approach, grasp, carry and release is not sufficient if its thresholds do not match the robot and scene. Conversely, numerical tuning cannot repair a single-stage objective that confuses contact acquisition with task completion. This is directly relevant to humanoid manipulation, where each phase changes which contacts are desirable.

The 86.5% figure is a simulation macro-average, not a hardware success rate. The project page shows autonomous G1 executions, including repeated kicks, but one kick sequence uses motion-capture box pose. Other demonstrations use robot-mounted RGB-D input with off-board computation. The paper does not publish a broad table of repeated physical success counts comparable to the simulation study.

Limitations and availability

InterEvolve spends inference-time compute on LLM calls, CMA-ES and hundreds of simulator rollouts per candidate. It does not replan reward code in real time during hardware execution. Results are bounded by the pretrained motor repertoire, reward feature bank, available scene measurements and simulator fidelity. Transfer to different objects can require manual program adaptation; unchanged large-box programs show large drops on several objects.

This is an arXiv v1 preprint, and RoboSkin.ai did not run the simulator or robot. The official project publishes explanatory material and videos. The November 4 review did not verify an official implementation, trained-controller download, dataset release or software license through that project page. The manuscript is distributed under arXiv's perpetual non-exclusive license.

Keep reward search, prompt replay and feedback separate

Three different processes sit behind an InterEvolve hardware demonstration. The deployment appendix describes a latent sequence obtained from a simulated rollout and replayed step by step on the robot. The controller still responds to current proprioception and estimated object state. Thus replaying prompts is neither online LLM program revision nor blind playback of joint commands.

ProcessChanges during the processWhat stays fixed
Simulation searchProgram stages, constants and the external text libraryPretrained controller, source bank and task-verification criteria
Hardware prompt replayWhich stored latent prompt is supplied at each stepThe selected sequence and pretrained motor policy
Feedback executionMotor output based on current robot and object estimatesAvailable sensing, trained repertoire and controller interface

The verifier is critical: maximizing a reward is not enough if a robot can earn it while standing still. Independent task criteria score the actual outcome, and changing reward code must not redefine those criteria. Search and final evaluation use different scenes but the same bounded perturbation distribution; this is not an unrestricted generalization test.

For a proposed physical replication, record the prompt sequence, object mesh, pose estimator, confidence gate and timing. The paper's camera pipeline can pause when pose confidence is insufficient, and one kick demonstration uses motion capture. Geometric proximity features are not measured contact forces. Objects new to reward search may also have appeared in motor pretraining, so record novelty at both stages. These distinctions fit the humanoid observation-to-control map and prevent the simulated 86.5% result or one-family compute budget from becoming a hardware capability claim.

*Source review updated November 4, 2026. Original publication dates are retained. Results are reported by the authors; RoboSkin has not run these systems.*

Continue the topic

Robotics benchmarksNIST proposes a baseline benchmark for comparable humanoid robot capabilitiesHumanoid benchmarksHumanoidToolBench separates tool choice from task completionHumanoid contact controlReactive multi-contact planning selects a humanoid hand brace in about 10 ms