Abstract

This paper presents a review of the application of facial emotion recognition and eye tracking technology for mentally fit intensive speech, disabled persons. It also presents a review of the augmentative and alternative communication (AAC), already developed emotion recognition systems or frameworks and some current speech disability support devices selected from the internet. Employing these techniques, the design of simplex-based automatic real-time expressive communication system to facilitate effective interpretation of thoughts of speech-disabled is advocated in this paper. While several devices are available, some with eye-tracking capability, but there is still a need for more versatile, cost-effective device or framework and the challenge is to achieve speech like a human from AAC based device.

References

• “World Health Organization-Health topics-Disabilities” [Online] Available: https://www.who.int/topics/disabilities/en/
• “International Classification of Functioning, Disability and Health” [Online] Available: https://www.who.int/classifications/icf/en/
• “World report on disability” [Online] Available: https://en.wikipedia.org/wiki/World_report_on_disability
• Surbhi Rathi, Ujwalla Gawande, “Development of full-duplex communication system for deaf and dumb people”, 7th International conference on cloud computing, data science, and engineering-Confluence, IEEE 2017
• Pet Mirenda, Teresa lacono, “Communication Options for Persons with Severe and Profound Disabilities: State of the Art and Future Directions”, sage journals, Vol.15, issue: 1, pages: 3-21, March 1990, DOI: https://doi.org/10.1177/154079699001500102
• Barry M. prizant, Lisa R. Audet, Grace M. Burke, Lauren J. Hummel, Suzanne R. Maher, Geraldine theadore, “Communication Disorders and Emotional/Behavioral Disorders in Children and Adolescents”, Journal of speech and hearing disorders, May 1990, Vol.55,179-192, DOI:10.1044/jshd.5502.179
• “Emotion Recognition” [Online] Available: https://en.wikipedia.org/wiki/Emotion_recognition
• Salwa Said, Olfa Jemai, Mourad Zaied, Chokri Ben Amar, “Wavelet Networks for Facial Emotion Recognition”, 15th International Conference on Intelligent Systems Design and Applications (ISDA), IEEE 2015, DOI:10.1109/ISDA.2015.7489242
• Priya Saha, Debotosh Bhattacharjee, Barin Kumar De, MitaNasipuri, “An Approach to Detect the Region of Interest of Expressive Face Images”, International Conference on Information and Communication Technologies (ICICT 2014), Elsevier Procedia Computer Science 46(2015)17391746, DOI:10.1016/j.procs.2015.02.123
• Jyoti Kumari, R.Rajesh, K.M Pooja, “Facial expression recognition: A survey”, Second International Symposium on Computer Vision and the Internet (VisionNet’15), Elsevier Procedia Computer Science 58(2015), 486-491
• Ma Xiaoxi, Lin Weisi, Huang Dongyan, Dong Minghui, Haizhou Li, “Facial emotion recognition”, 2nd International Conference on Signal and Image Processing (ICSIP), IEEE, Aug 2017 DOI: 10.1109/SIPROCESS.2017.8124509
• Mohan Ghai, Shamit Lal, Shivam Duggal, Shrey Manik, “Emotion recognition on speech signals using machine learning”, International Conference on Big Data Analytics and Computational Intelligence (ICBDAC),2017, DOI: 10.1109/ICBDACI.2017.8070805
• Roshan Jameel, Abhishek Singhal, Abhay Bansal, “A Comprehensive study on facial expressions recognition techniques”, 6th International Conference –Cloud System and Big data Engineering (Confluence), IEEE2016, DOI:10.1109/CONFLUENCE.2016.7508167
• Ligang Zhang, Dian Tjondronegoro, “Facial expression recognition using facial movement features ”, IEEE Transactions on Affective Computing, Vol.2, Issue No.4,2011, DOI: 10.1109/T-AFFC.2011.13
• Shoaib Kamal, Dr. Farrukh Sayeed, Mohammed Rafeeq, “Facial Emotion Recognition for Human-Computer Interactions using hybrid feature extraction technique”, International Conference on Data Mining and Advanced Computing (SAPIENCE), IEEE, March 2016, DOI: 10.1109/SAPIENCE.2016.7684129
• Pawel Tarnowski, Marcin Kolodziej, Andrzej Majkowski, Remigiusz J. Rak, “Emotion recognition using facial expressions”, International Conference on Computational Science, ICCS 2017,12-14 June 2017, Zurich Switzerland, DOI: https://doi.org/10.1016/j.procs.2017.05.025
• Md. Zia Uddin, Weria Khaksar, Jim Torresen,“ Facial expression recognition using salient features and convolutional neural network”, Vol. 5, IEEE Access, Nov 2017, DOI: 10.1109/ACCESS.2017.2777003
• Pramodini A. Punde, Mukti E. Jadhav, Ramesh R. Manza, “A study of eye tracking technology and its applications”, 1st International Conference on Intelligent Systems and Information Management (ICISIM),  IEEE, Oct 2017, DOI: 10.1109/ICISIM.2017.8122153
• Amer Al-Rahayfeh, Miad Faezipour, “Eye Tracking and Head Movement Detection: A State-of-Art Survey”, IEEE Journal of Translational Engineering in Health and Medicine (Volume: 1), DOI:10.1109/JTEHM.2013.2289879
• Shaun K. Kane, Meredith Ringel Morris, Ann Paradiso, Jon Campbell, “At times avuncular and cantankerous, with the reflexes of a mongoose: Understanding Self-Expression through Augmentative and Alternative Communication Devices”, Proceedings of CSCW 2017, ACM, DOI: http://dx.doi.org/10.1145/2998181.2998284
• Alexander Fiannaca, John Campbell, Ann Paradiso, Meredith Ringel Morris, “Voice setting: Voice Authoring Uis for Improved Expressivity in Augmentative Communication”, Proceedings of CHI 2018,ACM,DOI:https://doi.org/10.1145/3173574.3173857
• Christos D. Katsis, Nikolaos S. Katersidis, Dimitrios I. Fotiadis, “An integrated system based on physiological signals for the assessment of affective states in patients with anxiety disorders”, Biomedical Signal Processing and Control 6(2011)261-268, Elsevier, DOI:10.1016/j.bspc.2010.12.001
• M.T.Quazi, S.C. Mukhopadhyay, N.K.Suryadevara, Y.M. Huang, “Towards the Smart Sensors Based Human Emotion Recognition”, International Instrumentation and Measurement Technology Conference (12 MTC), IEEE 2012, DOI: 10.1109/12MTC.2012.6229646
• Eun-Hye Jang, Byoung-Jun Park, Mi-SookPark, SngHyeobKim, Jin Han Sohn, “Analysis of physiological signals for recognition of boredom, pain and surprise emotions”, Journal of Physiological Anthropology (2015)34:25
• Kathawut Rojanasaroch, Terravisit Laohapensaeng, “Communication Aid Device for Illness Deaf-Mute”, 12th International Conference on Electrical Engineering/Electronics, Computer, Telecommunications and Information Technology (ECTI-CON), IEEE2015, DOI:10.1109/ECTICon.2015.7207127
• “TobiiDynavox I-Series+” [online] Available: https://www.tobii.com/group/news-media/press-releases/tobii-dynavox-launches-the-new-i-series-and-communicator-5-for-even-more-efficient-communication/
• Mingmin Zhao, Fadel Adib, Dina Katabi, “Emotion recognition using wireless signals”, MobiCom’16 Proceedings of the 22nd Annual International Conference on Mobile Computing and Networking, ACM, Oct 2016
• Elena Simion, “Augmentative and alternative communication - support for people with severe speech disorders”, Procedia – Social and Behavioral Sciences, Vol. 128, April 2014, Pages 77-81, DOI: https://doi.org/10.1016/j.sbspro.2014.03.121
• Janice Light and David McNaughton (2012), “The Changing Face of Augmentative and Alternative Communication: Past, Present, and Future Challenges”, Augmentative and Alternative Communication, 28:4, 197-204, DOI: 10.3109/07434618.2012.737024
• Jeff Sigafoos, Mark F. O. Reilly, Giulio E. Lancioni, Dean Sutherland, “Augmentative and Alternative Communication for Individuals with Autism Spectrum Disorder and Intellectual Disability”, Springer International Publishing, Jan 2014, Online ISSN: 2196-2987, DOI: https://doi.org/10.1007/s40474-013-0007-x
• “Attainment Go Talk Pocket” [Online] Available: https://www.enablemart.com/attainment-gotalk-pocket
• “Logan Prox Talker” [Online] Available: https://logantech.com/products/proxtalker
• “MegaBee Eye Pointing Communication Tablet” [Online] Available: https://logantech.com/products/megabee
• “Beamz Interactive Music System” [Online] Available: https://www.inclusivetlc.com/beamz-interactive-music-system
• “Tobii Dynavox I-Series+” [Online] Available: https://www.tobiidynavox.com/devices/eye-gaze-devices/i-15-with-communicator/
• G.J. Edwards, C.J. Taylor, T.F. Cootes, “Interpreting face images using active appearance models”, Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition,1998, DOI:10.1109/AFGR.1998.670965
• “ITU-T Rec. P 800 (08/96) Methods for subjective determination of transmission quality” [Online] Available: https://www.itu.int/rec/T-REC-P.800-199608-I/en
• Viswanathan Mahesh and Viswanathan Madhubalan (2005), “Measuring speech quality for text-to-speech systems: Development and assessment of a modified mean opinion score (MOS) scale”, Computer Speech and Language-Elsevier, 19(1): 55-83, DOI:10.1016/j.csl.2003.12.001.
• Black Alan W. and Tokuda Keiichi, “The Blizzard Challenge 2005: Evaluating corpus-based speech synthesis on common datasets”, Conference: INTERSPEECH 2005,77-80 - Eurospeech, 9th European Conference on Speech Communication and Technology, Lisbon, Portugal, September 4-8, 2005
• Yu Yun Chang, “Evaluation of TTS systems in intelligibility and comprehension tasks”, ROCLING '11 Proceedings of the 23rd Conference on Computational Linguistics and Speech Processing Pages 64-78, Taipei, China-September 08-09, 2011