Research Article
Towards a Smart Wearable Tool to Enable People with SSPI to Communicate by Sentence Fragments
@INPROCEEDINGS{10.1007/978-3-319-11564-1_10, author={Gyula V\o{}r\o{}s and Anita Verő and Bal\^{a}zs Pint\^{e}r and Brigitta Miksztai-R\^{e}they and Takumi Toyama and Andr\^{a}s Lőrincz and Daniel Sonntag}, title={Towards a Smart Wearable Tool to Enable People with SSPI to Communicate by Sentence Fragments}, proceedings={Pervasive Computing Paradigms for Mental Health. 4th International Symposium, MindCare 2014, Tokyo, Japan, May 8-9, 2014, Revised Selected Papers}, proceedings_a={MINDCARE}, year={2014}, month={12}, keywords={Augmentative and alternative communication Smart glasses Eye tracking Head-mounted display Speech synthesis Natural language processing Language models}, doi={10.1007/978-3-319-11564-1_10} }- Gyula Vörös
Anita Verő
Balázs Pintér
Brigitta Miksztai-Réthey
Takumi Toyama
András Lőrincz
Daniel Sonntag
Year: 2014
Towards a Smart Wearable Tool to Enable People with SSPI to Communicate by Sentence Fragments
MINDCARE
Springer
DOI: 10.1007/978-3-319-11564-1_10
Abstract
The ability to communicate with others is of paramount importance for mental well-being. In this paper, we describe an interaction system to reduce communication barriers for people with severe speech and physical impairments (SSPI) such as cerebral palsy. The system consists of two main components: (i) the head-mounted human-computer interaction (HCI) part consisting of smart glasses with gaze trackers and text-to-speech functionality (which implement a communication board and the selection tool), and (ii) a natural language processing pipeline in the backend in order to generate complete sentences from the symbols on the board. We developed the components to provide a smooth interaction between the user and the system thereby including gaze tracking, symbol selection, symbol recognition, and sentence generation. Our results suggest that such systems can dramatically increase communication efficiency of people with SSPI.