Tactile-JEPA pretrains electronic-skin encoders on taxel topology
Tactile-JEPA learns sensor-specific encoders with graph-based masking. Review the original perception and policy results separately from a later repository training recipe.
Researchers from Sber AI, Skolkovo Institute of Science and Technology, HSE University and the Artificial Intelligence Research Institute released Tactile-JEPA as an arXiv preprint on September 21, 2026. The method pretrains encoders for distributed tactile sensors by masking connected and dispersed regions of an irregular taxel graph, rather than forcing electronic-skin signals onto a camera-like pixel grid. Paper and version record.
Explore tactile sensing for the broader comparison.
Key takeaways
- Tactile-JEPA learns from unlabeled time windows and known taxel connectivity, predicting hidden taxel embeddings from visible context with both local and global graph masks.
- Across three public datasets, it reports a 6.3% reduction in force RMSE and a 20.8% reduction in in-hand orientation RMSE against the strongest comparison for those metrics.
- It is not best on every original-paper downstream result. On DECO-50 policy learning, MAE pretraining records 0.4400 normalized-action RMSE versus 0.4634 for Tactile-JEPA; lower is better. Tables II and III.
What changed
Distributed electronic skins produce multivariate time series from taxels placed across hands, feet or bodies. Their geometry is often sparse and irregular. Tactile-JEPA represents those taxels as a connectivity graph. A local target mask grows from a seed along connected neighbors; a global mask samples across the surface. The model receives the remaining context and predicts embeddings for the hidden regions. Equal local and global masking is intended to preserve fine contact details and whole-surface state.
The encoder uses 192-dimensional embeddings and three attention heads, with 12 blocks for Xela/DECO and eight for the tactile-socks experiments. The paper freezes pretrained encoders for most perception tests, then trains task-specific heads. Pretraining used four NVIDIA A100 80 GB GPUs and took 0.7 to 6.1 hours depending on the dataset and task. Architecture and compute.
Three datasets, three evidence types
| Dataset | Sensor and scale | Evaluated use |
|---|---|---|
| Sparsh-skin | Xela uSkin, 368 three-axis taxels, 100 Hz, 4.6 hours | Object class, force and in-hand pose |
| Tactile socks | Two piezoresistive socks, 453 total taxels, 14 Hz, 5.9 hours | Human action and full-body pose |
| DECO-50 subset | Two Inspire hands, 2,124 taxels, 30 Hz, 16.6 hours | Visuo-tactile assembly policy |
On Sparsh-skin, Tactile-JEPA reports total force RMSE of 14.05 cN versus 15.00 cN for end-to-end training, the strongest alternative in that row. Orientation RMSE is 5.48 degrees versus 6.92 degrees for BYOL. On tactile socks, action accuracy is 96.91% versus 95.63% for DINO, and full-body pose RMSE is 13.42 radians versus 13.78. These are means across repeated training runs, and selected top-two comparisons use a one-sided Welch test with three pretraining seeds and no multiple-comparison correction.
The original policy table requires a different reading. Tactile-JEPA improves over vision-only, random tactile embeddings and an end-to-end tactile encoder, but MAE is better on the reported DECO-50 action RMSE. The paper's stronger claim is stable performance across heterogeneous datasets: BYOL or MAE collapses on some datasets, while Tactile-JEPA trains under one mask configuration.
What this means for robot skin
RoboSkin analysis: representing taxel layout explicitly is a sound fit for conformable robot skin, where physical neighbors need not share adjacent array indices. The local-global mask result also suggests that pretraining should preserve both a contact patch and the state of the wider surface. This is relevant to tactile foundation models, but the evidence is still sensor-family-specific rather than a universal encoder.
Integration is not plug-and-play. DECO-50 groups four or five neighboring taxels to make attention over 2,124 channels affordable. Magnetic data receives baseline correction; piezoresistive data uses different normalization. Dataset coordinates are also missing for DECO-50, so grouping and topology construction must be supplied by the implementation. The datasets directory can help compare these data contracts before reuse.
Limitations and availability
This is a preprint, and RoboSkin.ai has not rerun the experiments. All headline improvements are within the selected datasets, task heads and splits. The policy score is imitation-action error, not a physical task success rate. A lower action RMSE also does not by itself prove safer or more reliable contact.
The paper links a public repository. The November 4 review supersedes the earlier README-access description: the repository now documents Tactile-JEPA datasets, preparation, training presets and downstream evaluation. Its pretrained-model entry remains TBD. The license at the reviewed revision is Creative Commons Attribution-NonCommercial 4.0; third-party datasets and assets retain their own terms. Documentation was reviewed, but no training, download or checkpoint execution was performed.
Reuse the training recipe without assuming a universal checkpoint
Tactile-JEPA needs known taxel connectivity to sample masks during pretraining, but does not require numerical three-dimensional taxel coordinates. The graph is not needed by the downstream encoder; learned taxel identities and the expected input ordering still are. That distinction does not establish transfer to a different sensor layout without adaptation. The paper trains separate encoders for the datasets and, for the socks, separate task settings. It tests a reusable method rather than one checkpoint shared unchanged across all sensors. Method and evaluation.
| Reuse question | Evidence to retain |
|---|---|
| Which taxel is this? | Channel order, connectivity, grouped cells, baseline and normalization conventions |
| How much time is represented? | Ten frames at 100 Hz, three at 30 Hz or five at 14 Hz are different input windows |
| What is predicted? | Masked embeddings during pretraining; task-specific targets after training a head |
| What generalization is tested? | Whole demonstrations, objects, sessions and sensor layouts are different split units |
| What does the policy metric mean? | Offline error over twelve hand-action dimensions and sixteen future steps; not robot task completion |
The Sparsh-skin classification split partitions trajectories within the collected object set, rather than establishing unseen-object recognition. DECO splits whole demonstrations. Short-window representation learning should also not be confused with forecasting future tactile observations; a downstream head can consume a longer sequence of encoded windows.
The current pinned README now documents a separate, revised DECO training recipe. It reports action RMSE 0.43880 ± 0.00185 against MAE 0.43996 ± 0.00288. These are author-reported follow-up results, not replacements for the original paper's table. The spread is the standard deviation across twelve heads from three pretraining seeds, not a confidence interval or twelve independent encoder trainings. The numerical difference alone does not establish a statistically reliable advantage.
A receiving experiment should pin the recipe, split and checkpoint together. The README also identifies exploratory mixed-source socks presets that include held-out pose splits; those are not interchangeable with train-only task baselines. The tactile AI comparison keeps representation error separate from physical control evidence.
Sources
- Tactile-JEPA arXiv record, submitted September 21, 2026
- Tactile-JEPA v1 methods, datasets and results
- Author-linked code repository and license
*Source review updated November 4, 2026. Original publication dates are retained. Research results are author-reported; RoboSkin has not run these systems or reproduced their experiments.*
Continue the topic
