Gesture detection is the primary and most significant step for sign language detection and sign language is the communication medium for people with speaking and hearing disabilities. This paper presents a novel method for dynamic hand gesture detection using Hidden Markov Models (HMMs) where we detect different English alphabet letters by tracing hand movements. The process involves skin color-based segmentation for hand isolation in video frames, followed by morphological operations to enhance image trajectories. Our system employs hand tracking and trajectory smoothing techniques, such as the Kalman filter, to monitor hand movements and refine gesture paths. Quantized sequences are then analyzed using the Baum-Welch Re-estimation Algorithm, an HMM-based approach. A maximum likelihood classifier is used to identify the most probable letter from the test sequences. Our method demonstrates significant improvements over traditional recognition techniques in real-time, automatic hand gesture recognition, particularly in its ability to distinguish complex gestures. The experimental results confirm the effectiveness of our approach in enhancing gesture-based sign language detection to alleviate the barrier between the deaf and hard-of-hearing community and general people.
References
[1]
Baum, L.E. and Petrie, T. (1966) Statistical Inference for Probabilistic Functions of Finite State Markov Chains. The Annals of Mathematical Statistics, 37, 1554-1563. https://doi.org/10.1214/aoms/1177699147
[2]
Rabiner, L.R. and Juang, B.H. (1993) Fundamentals of Speech Recognition. Prentice Hall, New Jersey.
[3]
Lee, J.J. and Kim, J.H. (2001) Data-Driven Design of HMM Topology for Online Handwriting Recognition. International Journal of Pattern Recognition and Artifical Intelligence, 15, 107-121. https://doi.org/10.1142/S0218001401000769
[4]
Wilson, A.D. and Bobick, A.F. (2001) Hidden Markov Models for Modeling and Recognizing Gesture under Variation. International Journal of Pattern Recognition and Artificial Intelligence, 15, 123-160. https://doi.org/10.1142/S0218001401000812
[5]
Nefian, A. and Hayes, M. (1998) Hidden Markov Models for Face Recognition. Proceedings of the International Conference on Acoustics, Speech, and Signal Processing (ICASSP98), Seattle, 12-15 May 1998, 2721-2724.
[6]
Xu, D.Y. (2006) A Neural Network approach for Hand Gesture Recognition in Virtual Reality Driving Training System of SPG. Proceedings of the 18th International Conference on Pattern Recognition (ICPR ‘06), Hong Kong, 20-24 August 2006, 519-522. https://doi.org/10.1109/ICPR.2006.109
[7]
Hsieh, C.-C., Liou, D.-H. and Lee, D. (2010) A Real Time Hand Gesture Recognition System Using Motion History Image. 2010 2nd International Conference on Signal Processing Systems, Dalian, 5-7 July 2010, V2-394-V2-398. https://doi.org/10.1109/ICSPS.2010.5555462
[8]
Holden, E., Owens, R. and Roy, G. (1996) Hand Movement Classification Using an Adaptive Fuzzy Expert System. International Journal of Expert Systems, 9, 465-480.
[9]
Elmezain, M., Al-Hamadi, A. and Michaelis, B. (2008) Real-Time Capable System for Hand Gesture Recognition Using Hidden Markov Models in Stereo Color Image Sequences. Journal of WSCG, 16, 65-72.
[10]
Wang, X.Y., Xia, M., Cai, H.W., Gao, Y. and Cattani, C. (2012) Hidden-Markov-Models-Based Dynamic Hand Gesture Recognition. Mathematical Problems in Engineering, 2012, Article ID: 986134. https://doi.org/10.1155/2012/986134
[11]
Liu, N.J., Lovell, B.C., Kootsookos, P.J. and Davis, R.I.A. (2004) Model Structure Selection & Training Algorithms for an HMM Gesture Recognition System. Ninth International Workshop on Frontiers in Handwriting Recognition, Tokyo, 26-29 October 2004, 100-105.
[12]
Huang, S.L. and Hong, J.X. (2011) Moving Object Tracking System Based on Camshift and Kalman Filter. 2011 International Conference on Consumer Electronics, Communications and Networks (CECNet), Xianning, 16-18 April 2011, 1423-1426. https://doi.org/10.1109/CECNET.2011.5769081
[13]
Liu, N.J., Lovell, B.C. and Kootsookos, P.J. (2003) Evaluation of HMM Training Algorithms for Letter Hand Gesture Recognition. Proceedings of the 3rd IEEE International Symposium on Signal Processing and Information Technology (IEEE Cat. No. 03EX795), Darmstadt, 17 December 2003, 648-651.
[14]
Nahar, L., Zabu, Z.A., Raihan, A., Islam, M.I. and Emon, M.I. (2022) A Comparative Selection of Best Activation Pair Layer in Convolution Neural Network for Sentence Classification Using Deep Learning Model. 2022 7th International Conference on Communication and Electronics Systems (ICCES), Coimbatore, 22-24 June 2022, 1398-1404. https://doi.org/10.1109/ICCES54183.2022.9835806
[15]
Alam, S.U., Uddin, R., Alam, M.J., Raihan, A., Mahtab, S.S. and Bhowmik, S. (2022) Mathematical Modeling and Performance Evaluation of 3D Ferroelectric neGative Capacitance FinFET. Modelling and Simulation in Engineering, 2022, Article ID: 8345513. https://doi.org/10.1155/2022/8345513
[16]
Minhajul Alam, S.M., Barua, A., Raihan, A., Alam, M.J., Chakma, R., Mahtab, S.S. and Biswas, C. (2021) Design and Implementation of a Smart Helmet System for Underground Miner’s Safety. In: Bindhu, V., Tavares, J.M.R.S., Boulogeorgos, A.A.A. and Vuppalapati, C., Eds., International Conference on Communication, Computing and Electronics Systems, Springer, Singapore, 301-311. https://doi.org/10.1007/978-981-33-4909-4_22
[17]
Alam, M.J., Mahbub, T., Rahman, M., Anonto, R.A., Raihan, A. and Mahtab, S.S. (2021) Effect of Anti Reflecting Coating (ARC) in GaAs Based P-I-N Solar Device with Gap as Window Layer. AIP Conference Proceedings, 2327, Article ID: 020046. https://doi.org/10.1063/5.0040004
[18]
Ali, F.B., Chakma, R., Raihan, A., Hasan, M., Khan, M.F., Alam, M.J., Akter, R. and Mahtab, S.S. (2021) Design and Implementation of a Time-Based Sun Tracking Solar System. AIP Conference Proceedings, 2327, Article ID: 020024.
[19]
Mahtab, S.S., Akter, R., Mahbub, T., Raihan, A., Anonto, R.A. and Alam, M.J. (2021) Impact Analysis of Anti-Reflection Coating on P-i-N Solar Device. In: Bindhu, V., Tavares, J.M.R.S., Boulogeorgos, A.A.A. and Vuppalapati, C., Eds., International Conference on Communication, Computing and Electronics Systems, Springer, Singapore, 279-287. https://doi.org/10.1007/978-981-33-4909-4_20
[20]
Akter, R., Mahin, M.M., Mahbub, T., Nahreen, K., Raihan, A., Chakma, R., Alam, M.J. and Mahtab, S.S. (2021) Solar Cells: Varieties and Utilization—A Short Review. AIP Conference Proceedings, 2327, Article ID: 020023. https://doi.org/10.1063/5.0039501
[21]
Hossain, M.D.I., Akter, R., Hasan, M., Raihan, A., Khan, M.F., Ali, F.B., Chakma, R., Alam, M.J. and Mahtab, S.S. (2021) Generation of Electrical Energy Using Road Transport Pressure at Speed Breaker. AIP Conference Proceedings, 2327, Article ID: 020025. https://doi.org/10.1063/5.0039999
[22]
Raihan, A. (2023) Structural, Optical, and Electrical Properties of WSe2 Thin Films Synthesized by Magnetron Sputtering. Master’s Thesis, Morgan State University, Baltimore.
[23]
Raihan, A., Jyotsna, D., Ravinder, K. and Dereje, S. (2023) Controlled Growth and Transport Property of Bi2Se3 and WSe2 Thin Film. APS March Meeting Abstracts, 2023, SS01-007.
