cc 15(3): e1

Research Article

A System for Multimodal Interaction with Kinect-Enabled Virtual Windows

Download1419 downloads
  • @ARTICLE{10.4108/icst.intetain.2015.259498,
        author={Ana M. Bernardos and \^{I}\`{o}igo Marqu\^{\i}nez and David G\^{o}mez and Juan Besada and Jos\^{e} Casar},
        title={A System for Multimodal Interaction with Kinect-Enabled Virtual Windows},
        journal={EAI Endorsed Transactions on Collaborative Computing},
        keywords={multimodal interaction, virtual windows, deictic interaction, smart spaces, smart objects},
  • Ana M. Bernardos
    Íñigo Marquínez
    David Gómez
    Juan Besada
    José Casar
    Year: 2015
    A System for Multimodal Interaction with Kinect-Enabled Virtual Windows
    DOI: 10.4108/icst.intetain.2015.259498
Ana M. Bernardos1,*, Íñigo Marquínez1, David Gómez1, Juan Besada1, José Casar1
  • 1: Universidad Politécnica de Madrid
*Contact email:


Commercial off-the-shelf gaming devices (e.g. such as Kinect) are demonstrating to have a great potential beyond their initial service purpose. In particular, when integrated within the environment or as part of smart objects, peripheral COTS for gaming may facilitate the definition of novel interaction methods, particularly applicable to smart spaces service concepts. In this direction, this paper describes a system prototype that makes possible to deliver multimodal interaction with the media contents in a Virtual Window. Using a Kinect device, the Interactive Window itself adjusts the video clipping to the real time perspective of the user – who can freely move within the sensor coverage are. On the clipped video, the user is able to select objects by pointing at meaningful image sections and to initiate actions related to them. Voice orders may also complete the interaction when necessary. Although implemented for smart spaces, the service concept can also be applied to learning, remote control processes or teleconference.