Research Article
Millimeter Wave Radar Sensing Technology for Filipino Sign Language Recognition
@INPROCEEDINGS{10.1007/978-3-031-34586-9_19, author={Jorelle Aaron Herrera and Almira Astrid Muro and Philip Luis Tuason III and Paul Vincent Alpano and Jhoanna Rhodette Pedrasa}, title={Millimeter Wave Radar Sensing Technology for Filipino Sign Language Recognition}, proceedings={Pervasive Computing Technologies for Healthcare. 16th EAI International Conference, PervasiveHealth 2022, Thessaloniki, Greece, December 12-14, 2022, Proceedings}, proceedings_a={PERVASIVEHEALTH}, year={2023}, month={6}, keywords={Sign Language Recognition Millimeter Wave Deep Learning}, doi={10.1007/978-3-031-34586-9_19} }- Jorelle Aaron Herrera
Almira Astrid Muro
Philip Luis Tuason III
Paul Vincent Alpano
Jhoanna Rhodette Pedrasa
Year: 2023
Millimeter Wave Radar Sensing Technology for Filipino Sign Language Recognition
PERVASIVEHEALTH
Springer
DOI: 10.1007/978-3-031-34586-9_19
Abstract
Filipino Sign Language (FSL) is the primary language used by the Deaf and Hard-of-Hearing (DHH) community in the Philippines. The lack of support for FSL from the government has led to a huge communication gap between the DHH and the hearing society. A substantial amount of research has been done to develop sign language recognition systems based on computer vision or wearable technology. However, most such systems are limited to controlled settings, while wearable systems also raise issues such as inconvenience to users. Millimeter wave (mmWave) technology has recently seen potential in gesture recognition applications as it allows the system to be non-contact and resistant to environmental factors while ensuring high resolution for recognizing small movements. An mmWave-based FSL recognition system that can translate isolated signs into their equivalent gloss was developed. Data from a TI IWR1443 radar sensor was fed into a preprocessing algorithm and a deep learning model composed of multi-view 2D CNNs and LSTM. 4 models were trained based on a dataset of 24 FSL signs gathered with 3 native Deaf signers in 3 different environments. A total of 3240 samples were collected, resulting in a model that attained an overall peak accuracy of 94.9% and an average real-time recognition latency of about 2.01 s. The model’s performance is comparable to both existing FSL and mmWave systems, showing immense potential for future work on FSL recognition using mmWave.
