Abstract

There are over 12,000 deaf and hearing-impaired individuals in Libya, according to 2018 statistics from the Social Solidarity Fund. Despite this significant population, access to effective communication tools remains limited. Deep learning has revolutionized various domains, and its impact on the recognition and translation of sign languages is no exception. This paper explores the application of deep learning, particularly Long Short-Term Memory (LSTM) networks, in the context of Libyan Sign Language (LSL) recognition and translation, aiming to bridge communication barriers for the hearing-impaired community in Libya. The paper presents a novel dataset and a robust LSL recognition model based on LSTM architecture and key point extraction using MediaPipe Holistic. Furthermore, the real-time testing showcases the practicality of the proposed LSL recognition model, offering the potential for real-world applications to empower the deaf community. The proposed LSTM model achieves an impressive testing accuracy of 84% in recognizing LSL gestures and translating them into Spoken Arabic. This work is a critical milestone in enhancing accessibility and empowering the deaf community in Libya.

References

• World Health Organization, “Deafness and hearing loss,” WHO, Feb. 02, 2024. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/deafness-and-hearing-loss.
• “Human Rights of the Deaf,” WFD. https://wfdeaf.org/our-work/human-rights-of-the-deaf.
• T. Johnston and A. Schembri, “Australian Sign Language (Auslan),” Cambridge University Press,2007, doi: https://doi.org/10.1017/cbo9780511607479.
• K. Kozik, “Without Sign Language, Deaf People Are Not Equal,” Human Rights Watch, Sep. 23, 2019. [Online]. Available: https://www.hrw.org/news/2019/09/23/without-sign-language-deaf-people-are-not-equal.
• S. Dhulipala, F. F. Adedoyin, and A. Bruno, “Sign and Human Action Detection Using Deep Learning,” Journal of Imaging, vol. 8, no. 7, p. 192, Jul. 2022, doi: https://doi.org/10.3390/jimaging8070192.
• M. Al-Qurishi, T. Khalid, and R. Souissi, “Deep Learning for Sign Language Recognition: Current Techniques, Benchmarks, and Open Issues,” IEEE Access, vol. 9, pp. 126917–126951, 2021, doi: https://doi.org/10.1109/access.2021.3110912.
• M. Madhiarasan, and P. P. Roy, “A comprehensive review of sign language recognition: Different types, modalities, and datasets, “ arXiv preprint, Apr. 2022, Accessed: May 15, 2023. [Online]. Available: http://arxiv.org/abs/2204.03328.
• C. J. Sruthi and A. Lijiya, “SigNet: A deep learning based indian sign language recognition system,” Proceedings of the 2019 IEEE International Conference on Communication and Signal Processing, ICCSP 2019, pp. 596–600, Apr. 2019, doi: 10.1109/ICCSP.2019.8698006.
• M. M. Kamruzzaman, “Arabic Sign Language Recognition and Generating Arabic Speech Using Convolutional Neural Network,” Wirel Commun Mob Comput, vol. 2020, pp. 1–9, May 2020, doi: 10.1155/2020/3685614.
• A. Elhagry and Elrayes, Rawan Glalal, “Egyptian Sign Language Recognition Using CNN and LSTM,” arXiv (Cornell University), Jul. 2021, doi: https://doi.org/10.48550/arxiv.2107.13647.
• B. A. Al-Mohimeed, H. O. Al-Harbi, G. S. Al-Dubayan, and A. A. Al-Shargabi, “Dynamic Sign Language Recognition Based on Real-Time Videos,” International Journal of Online and Biomedical Engineering (iJOE), vol. 18, no. 01, pp. 4–14, Jan. 2022, doi: 10.3991/ijoe.v18i01.27581.
• M. S. Abdallah, G. H. Samaan, A. R. Wadie, F. Makhmudov, and Y.-I. Cho, “Light-Weight Deep Learning Techniques with Advanced Processing for Real- Time Hand Gesture Recognition,” Sensors, vol. 23, no. 1, p. 2, Dec. 2022, doi: 10.3390/s23010002.Nn
• V. G and K. Goyal, “Indian Sign Language Recognition Using Mediapipe Holistic,” Apr. 2023, Accessed: May 29, 2023. [Online]. Available: https://arxiv.org/abs/2304.10256v1.
• G. H. Samaan et al., “MediaPipe’s Landmarks with RNN for Dynamic Sign Language Recognition,” Electronics (Basel), vol. 11, no. 19, p. 3228, Oct. 2022, doi: 10.3390/electronics11193228.
• B. Y. AlKhuraym, M. M. Ben Ismail, and O. Bchir, “Arabic Sign Language Recognition using Lightweight CNN-based Architecture,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 4, 2022, doi: 10.14569/IJACSA.2022.0130438.
• Grishchenko Ivan and Bazarevsky Valentin, “MediaPipe Holistic — Simultaneous Face, Hand and Pose Prediction, on Device,” Dec. 10, 2020. https://ai.googleblog.com/2020/12/mediapipe-holistic-simultaneous-face.html.
• J. D. Kelleher, “Deep learning,” Cambridge, Ma: The Mit Press, 2019, p. 299.
• J. Long, A. Khaliq, and K. M. Furati, “Identification and prediction of time-varying parameters of COVID-19 model: a data-driven deep learning approach,” International Journal of Computer Mathematics, vol. 98, no. 8, pp. 1617–1632, May 2021, doi: https://doi.org/10.1080/00207160.2021.1929942.
• S. Subburaj and S. Murugavalli, “Survey on sign language recognition in context of vision-based and deep learning,” Measurement: Sensors, vol. 23, p. 100385, Oct. 2022, doi: 10.1016/j.measen.2022.100385.
• “gTTS — gTTS documentation,” gtts.readthedocs.io, https://gtts.readthedocs.io/en/latest/.
• B. Subramanian, B. Olimov, S. M. Naik, S. Kim, K.-H. Park, and J. Kim, “An integrated mediapipe-optimized GRU model for Indian sign language recognition,” Sci Rep, vol. 12, no. 1, p. 11964, Jul. 2022, doi: 10.1038/s41598-022-15998-7.
• K. A. Rahim, Md. Khaliluzzaman, S. I. Khan, and M. S. Kabisha, “Face and Hand Gesture Recognition Based Person Identification System using Convolutional Neural Network,” International Journal of Intelligent Systems and Applications in Engineering, vol. 10, no. 1, pp. 105–115, Mar. 2022, doi: 10.18201/ijisae.2022.273.
• B. Sundar and T. Bagyammal, “American Sign Language Recognition for Alphabets Using MediaPipe and LSTM,” Procedia Comput Sci, vol. 215, pp. 642–651, 2022, doi: 10.1016/j.procs.2022.12.066.
• S. Afaq and S. Rao, “Significance of Epochs on Training a Neural Network”, A Neural Network. International Journal of Scientific & Technology Research, vol. 9, pp. 485-488, 2020, [Online]. Available: www.ijstr.org.