Multimodal Tactile Perception: Fusing Touch with Thermal and Proximity Sensing
Published in IEEE Transactions on Robotics (October 2025): First comprehensive multimodal artificial skin system combining pressure, temperature, vibration, and proximity data.
Multimodal Tactile Perception: Fusing Touch with Thermal and Proximity Sensing
**Published in IEEE Transactions on Robotics - October 2025**
Introduction
Biological skin provides far more than pressure information - it senses temperature, vibration, slip, and even proximity. We present the first artificial skin that integrates all these modalities.
Sensor Suite
1. Tactile Array
- 10,000 taxels (tactile elements) per 100cm^2
- 0.01N force sensitivity
- 1kHz sampling rate per taxel
2. Thermal Array
- 100 thermopiles per 100cm^2
- Temperature range: -20 degreesC to 100 degreesC
- Accuracy: ±0.1 degreesC
- Response time: 10ms
3. Vibration Sensors
- 500 accelerometers per 100cm^2
- Frequency range: 10Hz to 10kHz
- Resolution: 0.001g
4. Proximity Array
- Capacitive sensors
- Range: 0.1mm to 10mm
- Resolution: 0.01mm
Sensor Fusion Architecture
Hardware Level
- Time-division multiplexing reduces wiring
- On-skin preprocessing ASIC
- Event-based data transmission
Software Level
- Kalman filtering for noise reduction
- Bayesian fusion for uncertainty handling
- Deep learning for pattern recognition
Demonstrated Capabilities
Material Identification
- 98.7% accuracy on 100 materials
- Classification based on texture + temperature + compliance
- Training on synthetic data, tested on physical objects
Slip Prediction
- 99.2% prediction accuracy
- 50ms advance warning
- Enables preventive grip adjustment
Object Recognition
- 95.3% accuracy on common objects
- Works without vision (in dark/occluded)
- Continuous learning from new objects
Benchmark Comparison
| Modality | Biological | Our System | Previous Artificial |
|----------|-----------|------------|-------------------|
| Pressure | [Y] | [Y] | [Y] only |
| Temperature | [Y] | [Y] | [N] |
| Vibration | [Y] | [Y] | Partial |
| Proximity | Hair-like | [Y] | [N] |
Power Consumption
Total power: 150mW per 100cm^2
- Tactile: 80mW
- Thermal: 40mW
- Vibration: 25mW
- Proximity: 5mW
Battery life: 8 hours continuous operation (500mAh)
Real-World Validation
Deployed on:
- **Prosthetic Hand**: Enabled hot/cold object discrimination
squared. **Industrial Gripper**: Reduced object damage by 73%
cubed. **Exploration Rover**: Rock classification without vision
Open Source Release
All sensor fusion algorithms released as open source:
https://github.com/roboskin-ai/multimodal-fusion