Intelligent Technologies for Interactive Entertainment. 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28–30, 2016, Revised Selected Papers

Research Article

project SENSE – Multimodal Simulation with Full-Body Real-Time Verbal and Nonverbal Interactions

Download132 downloads
  • @INPROCEEDINGS{10.1007/978-3-319-49616-0_28,
        author={Hossein Miri and Jan Kolkmeier and Paul Taylor and Ronald Poppe and Dirk Heylen},
        title={project SENSE -- Multimodal Simulation with Full-Body Real-Time Verbal and Nonverbal Interactions},
        proceedings={Intelligent Technologies for Interactive Entertainment. 8th International Conference, INTETAIN 2016, Utrecht, The Netherlands, June 28--30, 2016, Revised Selected Papers},
        proceedings_a={INTETAIN},
        year={2017},
        month={1},
        keywords={Virtual reality Motion capture Interactive conversational agents Mimicry Social skills Interpersonal sensemaking},
        doi={10.1007/978-3-319-49616-0_28}
    }
    
  • Hossein Miri
    Jan Kolkmeier
    Paul Taylor
    Ronald Poppe
    Dirk Heylen
    Year: 2017
    project SENSE – Multimodal Simulation with Full-Body Real-Time Verbal and Nonverbal Interactions
    INTETAIN
    Springer
    DOI: 10.1007/978-3-319-49616-0_28
Hossein Miri1,*, Jan Kolkmeier1,*, Paul Taylor,*, Ronald Poppe2,*, Dirk Heylen1,*
  • 1: University of Twente
  • 2: Utrecht University
*Contact email: h.miri@utwente.nl, j.kolkmeier@utwente.nl, p.j.taylor@lancaster.ac.uk, r.w.poppe@uu.nl, d.k.j.heylen@utwente.nl

Abstract

This paper presents a multimodal simulation system, project-SENSE, that combines virtual reality and full-body motion capture technologies with real-time verbal and nonverbal communication. We introduce the technical setup and employed hardware and software of a first prototype. We discuss the capabilities of the system for the investigation of and the effects of . We argue that this prototype lays the technological basis for further research in interpersonal and social skills, as well as the social and emotional consequences of nonverbal mimicry in sustained interactions.