7th International Conference on Pervasive Computing Technologies for Healthcare

Research Article

Gaze-contingent audio-visual substitution for the blind and visually impaired

Download12 downloads
  • @INPROCEEDINGS{10.4108/icst.pervasivehealth.2013.252018,
        author={Lukas Twardon and Hendrik Koesling and Andrea Finke and Helge Ritter},
        title={Gaze-contingent audio-visual substitution for the blind and visually impaired},
        proceedings={7th International Conference on Pervasive Computing Technologies for Healthcare},
        publisher={IEEE},
        proceedings_a={PERVASIVEHEALTH},
        year={2013},
        month={5},
        keywords={visual impairment spatial perception sensory substitution neuroplasticity eyetracking gaze-contingency eeg},
        doi={10.4108/icst.pervasivehealth.2013.252018}
    }
    
  • Lukas Twardon
    Hendrik Koesling
    Andrea Finke
    Helge Ritter
    Year: 2013
    Gaze-contingent audio-visual substitution for the blind and visually impaired
    PERVASIVEHEALTH
    ICST
    DOI: 10.4108/icst.pervasivehealth.2013.252018
Lukas Twardon1,*, Hendrik Koesling1, Andrea Finke1, Helge Ritter1
  • 1: Bielefeld University
*Contact email: ltwardon@techfak.uni-bielefeld.de

Abstract

A lot of effort has been made to advance technologies that improve blind and partially sighted people’s spatial perception. One common approach is to enhance or substitute vision by audition. Most sensory substitution systems, however, have not attached any importance to eye movements. But eye movements play an essential role in mental imagery even in the absence of visual input. Therefore, we propose a system for gaze-contingent auditory substitution of spatial vision. It is intended to be a mobile helper in everyday life of the visually impaired. The prototype we have developed combines eyetracking with depth measuring and sonification techniques. We carried out both a proof-of-concept study in complete darkness and an exploratory EEG study. Our findings indicate that gaze-contingent sensory substitution permits depth perception and leads to intermodal (audio-visual) processing in untrained subjects. Hence, as a result of neuroplasticity, the blind and visually impaired might learn to perceive gaze-dependent sound visually.