Architectural Innovations and Methodological Advancements in Tactile AI: A Comprehensive Paradigm for Robotic Perception

Author

Arya Sajeev, Prof. Riya K. Prakash, Prof (Dr). L. C. Manikandan


Abstract

 Robotic perception is moving away from systems that lean almost entirely on cameras toward architectures that fuse several senses at once, and touch is quickly becoming one of the most important additions. Tactile Artificial Intelligence, or Tactile AI, gives a robot the ability to work in cluttered, unpredictable settings by reading contact information directly rather than inferring it from a distance. Conventional manipulators built around rigid kinematic models and camera feeds struggle whenever a task calls for fine motor control, because quantities such as friction, slip, and surface micro-geometry are invisible to a lens once the hand closes around an object. This paper reviews the current landscape of Tactile AI in depth, working through the main methodological threads: touch representations learned without labels, techniques for transferring a model across different sensor hardware, architectures that combine vision and touch, and the pipelines used to move policies from simulation into the real world. The discussion also covers how hardware and software are being co-designed together, tracing a path from dense, camera-based tactile skins to lightweight, field-replaceable magnetic alternatives. Drawing on foundation models such as the Tactile Transformer family and shared benchmarks such as ObjectFolder, the paper assembles a picture of what a resilient, dexterous, next-generation robotic perception stack needs to look like.


Keywords

Tactile AI, Visuo-Tactile Fusion, Self-Supervised Learning, Sim-to-Real Transfer, Tactile Transformers, Robotic Manipulation, Cross-Sensor Generalization.



Full Text:

Download Paper PDF


References


  1. Higuera, C., Sharma, A., Bodduluri, C. K., Fan, T., Lancaster, P., Kalakrishnan, M., Kaess, M., Boots, B., Lambeta, M., Wu, T., & Mukadam, M. (2024). Sparsh: Self-Supervised Touch Representations for Vision-Based Tactile Sensing. Conference on Robot Learning (CoRL).
  2. Zhao, J., Ma, Y., Wang, L., & Adelson, E. H. (2024). Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Tasks. Conference on Robot Learning (CoRL).
  3. Huang, B., Wang, Y., Yang, X., Luo, Y., & Li, Y. (2024). 3D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensing. Conference on Robot Learning (CoRL).
  4. Guzey, I., Evans, B., Chintala, S., & Pinto, L. (2023). Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play. Conference on Robot Learning (CoRL).
  5. Church, A., Lloyd, J., Hadsell, R., & Lepora, N. F. (2022). Tactile Sim-to-Real Policy Transfer via Real-to-Sim Image Translation. Conference on Robot Learning (CoRL).
  6. Gao, R., Dou, Y., Li, H., Agarwal, T., Bohg, J., Li, Y., Fei-Fei, L., & Wu, J. (2023). The ObjectFolder Benchmark: Multisensory Learning with Neural and Real Objects. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  7. Gao, R., Si, Z., Chang, Y.-Y., Clarke, S., Bohg, J., Fei-Fei, L., Yuan, W., & Wu, J. (2022). ObjectFolder 2.0: A Multisensory Object Dataset for Sim2Real Transfer. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
  8. Yu, Z., Xu, W., Yao, S., Ren, J., Tang, T., Li, Y., Gu, G., & Lu, C. (2023). Precise Robotic Needle-Threading with Tactile Perception and Reinforcement Learning. Conference on Robot Learning (CoRL).
  9. Ai, B., Tian, S., Shi, H., Wang, Y., Tan, C., Li, Y., & Wu, J. (2024). RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packing. Robotics: Science and Systems (RSS).
  10. Oller, M., Berenson, D., & Fazeli, N. (2024). Tactile-Driven Non-Prehensile Object Manipulation via Extrinsic Contact Mode Control. Robotics: Science and Systems (RSS).
  11. Suresh, S., Si, Z., Mangelson, J. G., Yuan, W., & Kaess, M. (2022). ShapeMap 3-D: Efficient Shape Mapping Through Dense Touch and Vision. IEEE International Conference on Robotics and Automation (ICRA).
  12. Bauza, M., Bronars, A., & Rodriguez, A. (2022). Tac2Pose: Tactile Object Pose Estimation from the First Touch. ArXiv preprint / The International Journal of Robotics Research.
  13. Chen, Y., Sipos, A., Van der Merwe, M., & Fazeli, N. (2022). Visuo-Tactile Transformers for Manipulation. Conference on Robot Learning (CoRL).
  14. Si, Z., & Yuan, W. (2022). Taxim: An Example-Based Simulation Model for GelSight Tactile Sensors. IEEE Robotics and Automation Letters (RA-L).
  15. Wang, S., Lambeta, M., Chou, P.-W., & Calandra, R. (2022). TACTO: A Fast, Flexible, and Open-Source Simulator for High-Resolution Vision-Based Tactile Sensors. IEEE Robotics and Automation Letters (RA-L).
  16. Bhirangi, R., Hellebrekers, T., Majidi, C., & Gupta, A. (2021). ReSkin: Versatile, Replaceable, Lasting Tactile Skins. Conference on Robot Learning (CoRL).
  17. Yang, F., Ma, C., Zhang, J., Zhu, J., Yuan, W., & Owens, A. (2022). Touch and Go: Learning from Human-Collected Vision and Touch. Conference on Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track.
  18. Bhirangi, R., Pattabiraman, V., Erciyes, E., Cao, Y., Hellebrekers, T., & Pinto, L. (2024). AnySkin: Plug-and-Play Skin Sensing for Robotic Touch. NeurIPS Workshop on Touch Processing: From Data to Knowledge.

 

 

Share your valuable work from Social Media Buttons