Intelligent Hand Gesture Recognition System Using Machine Learning

Author

Ankitha Marvalli Ramesh, Ganga Basvaraj Kakamari, Gousiya Noorain, Madeeha Tazeen, Dr. Chethan Chandra S Basavaraddi


Abstract

 Communication can become difficult for bedridden, elderly, or physically disabled patients who may have limited speech or mobility. This project presents an intelligent hand gesture recognition system that uses computer vision and deep learning to convert predefined hand gestures into meaningful communication messages. The system uses MediaPipe Hands to detect hand landmarks from video frames and represents each frame using normalized landmark coordinates. Temporal sequences of these features are processed using a Long Short-Term Memory (LSTM) network to learn movement patterns and classify dynamic gestures. An initial development dataset containing six emergency-related gesture classes—Call, Doctor, Help, Hot, Lose, and Pain—was processed to generate 126 features per frame for a two-hand representation. The sequences were resampled to a fixed length of 70 frames and used for LSTM training and evaluation. The current model achieved 96.67% accuracy on the held-out test set of 60 sequences. A real-time webcam prototype was also implemented to display the recognized gesture, confidence, patient message, hand detection status, and system information. The current results demonstrate the feasibility of the proposed approach. The present dataset is an initial development dataset rather than the final dataset, and future work will focus on a larger and more diverse dataset, additional gesture classes, and richer features.



Keywords

Hand Gesture Recognition, Indian Sign Language, MediaPipe, LSTM, Deep Learning, Computer Vision, Assistive Communication.



Full Text:

Download Paper PDF


References


  1. Sebbah and F. Z. Chelali, “Using Machine Learning and Deep Learning for Static and Dynamic Hand Gesture Recognition,” 2024 International Conference on Advances in Electrical and Communication Technologies (ICAECOT), 2024, pp. 1–6, doi: 10.1109/ICAECOT62402.2024.10828747.
  2. Zhao, X. H. Li, J. C. D. Cruz, M. S. Verdadero, J. C. Centeno and J. M. Novelero, “Hand Gesture Recognition Based on Deep Learning,” 2023 International Conference on Digital Applications, Transformation & Economy (ICDATE), 2023, pp. 250–254, doi: 10.1109/ICDATE58146.2023.10248500.
  3. Kumar Sharma, A. S. Kumar, A. Nagpal, R. Singh, D. Reddy and A. Chadha, “Analysis of Machine Learning for Recognizing Hand Gestures,” 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), 2023, pp. 1844–1847, doi: 10.1109/ICACITE57410.2023.10182676.
  4. Deepika, S. Choudhary, S. Kumar and K. Srinivas, “Machine Learning-Based Approach for Hand Gesture Recognition,” 2023 International Conference on Disruptive Technologies (ICDT), 2023, pp. 264–268, doi: 10.1109/ICDT57929.2023.10150843.
  5. Asih, B. S. Hantono and I. Hidayah, “Exploring Models Approach for Real-Time Hand Gesture Recognition,” 2025 International Conference on Advancement in Data Science, E-learning and Information System (ICADEIS), 2025, pp. 1–7, doi: 10.1109/ICADEIS65852.2025.10933403.
  6. Mohanarathinam, K. G. Dharani, R. Sangeetha, G. Aravindh and P. Sasikala, “Study on Hand Gesture Recognition by using Machine Learning,” 2020 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), 2020, pp. 1498–1501, doi: 10.1109/ICECA49313.2020.9297513.
  7. Zhang et al., “MediaPipe Hands: On-device Real-time Hand Tracking,” arXiv:2006.10214, 2020.
  8. Hochreiter and J. Schmidhuber, “Long Short-Term Memory,” Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.
  9. “A Video Dataset of the Hand Gestures of Indian Sign Language Words used in Emergency Situations,” Mendeley Data, DOI: 10.17632/2vfdm42337.1.

 

 

Share your valuable work from Social Media Buttons