SpectRobot turns single-point vibration into tactile policy input
Wormsensing and Hugging Face researchers encode remote vibration sensors as spectrograms for a robot policy. Longer history mattered more than extending the tested band from 10 to 100 kHz.
Joseph Rigal, Emmanuel Virot and Caroline Pascal introduced SpectRobot in an arXiv preprint submitted on September 21, 2026. The Wormsensing and Hugging Face researchers convert a single mechanically coupled tactile signal into a fixed-size time-frequency spectrogram, then feed it through a standard image encoder alongside camera observations. The result is a practical test of whether temporal bandwidth can sometimes substitute for dense spatial coverage. Paper and version record.
Key takeaways
- SpectRobot samples signals as fast as 200 kS/s, but converts each history window into a 224 by 224 grayscale spectrogram delivered to the policy at roughly 30 Hz.
- In the controlled four-class sorting task, a vision-only policy achieved 23% success. Tactile configurations ranged from 48% to 82% across sensor types, with 80 physical trials per condition.
- Longer temporal history was the stronger result: roughly 0.3-second windows stayed near chance, while 2.3 to 2.9-second windows reached about 86% to 92%. The 10 and 100 kHz confidence intervals overlapped. Methods and bandwidth study.
What changed
Most robot touch systems add more sensing locations. SpectRobot instead asks how much interaction information can be recovered from one sensor mounted away from the contact surface but mechanically coupled to the gripper. Successive vibration samples are transformed into power-spectral-density images with a linear frequency axis. An Action Chunking with Transformers policy uses ResNet-18 encoders for the top camera, wrist camera and tactile spectrogram.
The physical task uses an SO-101 arm to pick up and shake opaque boxes. Four visually identical classes contain nothing, one plastic spacer, seven plastic spacers or seven metal nuts. A trial counts only if the robot grasps the box and deposits it in the correct bin. Each training dataset contains 40 demonstrations; each sensor condition is evaluated with 20 trials per class. Experimental protocol.
What the sensor comparison shows
The reported success rates are 82% for a MEMS accelerometer, 80% at best for the IEPE Dragonfly strain sensor, 73% for a PZT disk, 71% for an IEPE accelerometer, 67% for a passive Dragonfly, 58% for a metallic strain gauge and 48% for an IEPE load cell. The authors explicitly note that several 95% Wilson confidence intervals overlap, so this is not a statistically resolved hardware leaderboard.
The temporal ablation is more informative. At 10 kHz, success rose from about 31% with 0.36 seconds of history to about 86% with 2.9 seconds. At 100 kHz, it rose from about 32% with 0.29 seconds to about 92% with 2.3 seconds. Bandwidth and duration were not independently controlled because the same 224-pixel input grid was used. The safe conclusion is that longer context helped this shaking task; it is not that 100 kHz is generally unnecessary.
The authors also tested a roughly EUR 100 acquisition chain built around an ADS8688 converter and Teensy 4.1. Three sensors reached about 90% on a separately collected, comparable dataset. That is encouraging engineering evidence, but not a matched replacement test against the research-grade chain.
What this means for robotics
RoboSkin analysis: SpectRobot offers a useful design alternative when abrasion, wiring or surface geometry makes a dense array difficult. It does not recover contact location from one channel, and piezoelectric measurements emphasize dynamics rather than true static load. A real system may still combine these signals with distributed tactile sensors, motor-current force estimates or robot skin.
The image-like interface may reduce software friction because it fits existing vision encoders and the LeRobot workflow. Engineers still need synchronized acquisition, anti-alias filtering, stable mechanical coupling and dataset-level calibration checks. The tactile calibration guide explains why a shared tensor shape does not make different sensor installations interchangeable.
Limitations and availability
The study is a preprint and RoboSkin.ai has not reproduced it. It evaluates one sorting task, one arm family and independently collected datasets for several grippers. Raw time signals were not retained in the reported pipeline, which limits reprocessing with alternative windows. Two authors work for Wormsensing, the Dragonfly sensor manufacturer; the paper discloses this conflict.
The paper states that code is available under Apache 2.0 and points to the SpectRobot GitHub organization. As inspected on September 22, that organization exposed an Apache-2.0 project website repository, but no separate training-code repository was visible. The linked Hugging Face profile did expose many named datasets and policy models, including September 21 task assets, but not a single curated release page. Readers should identify the exact dataset and checkpoint before attempting reproduction.
Sources
Continue the topic
