Research Article
Virtual Reality and Audiovisual Experience in the AudioVirtualizer
@ARTICLE{10.4108/eai.19-3-2021.169037, author={Adinda van ’t Klooster and Nick Collins}, title={Virtual Reality and Audiovisual Experience in the AudioVirtualizer}, journal={EAI Endorsed Transactions on Creative Technologies}, volume={8}, number={27}, publisher={EAI}, journal_a={CT}, year={2021}, month={3}, keywords={Virtual Reality, Audiovisuals, Machine Listening, Interactive Art}, doi={10.4108/eai.19-3-2021.169037} }- Adinda van ’t Klooster
Nick Collins
Year: 2021
Virtual Reality and Audiovisual Experience in the AudioVirtualizer
CT
EAI
DOI: 10.4108/eai.19-3-2021.169037
Abstract
INTRODUCTION: Virtual Reality (VR) provides new possibilities for interaction, immersiveness, and audiovisual combination, potentially facilitating novel aesthetic experiences.
OBJECTIVES: In this project we created a VR AudioVirtualizer able to generate graphics in response to any sound input in a visual style similar to a body of drawings by the first author.
METHODS: In order to be able to make the system able to respond to any given musical input we developed a Unity plugin that employs real-time machine listening on low level and medium-level audio features. The VR deployment utilized SteamVR to allow the use of HTC Vive Pro and Oculus Rift headsets.
RESULTS: We presented our system to a small audience at PROTO in Gateshead in September 2019 and observed people’s preferred ways of interacting with the system. Although our system can respond to any sound input, for ease of interaction we chose four previously created audio compositions by the authors of this paper and microphone input as a restricted set of sound input options for the user to explore.
CONCLUSION: We found that people’s previous experience with VR or gaming influenced how much interaction they used in the system. Whilst it was possible to navigate within the scenes and jump to different scenes by selecting a 3D sculpture in the scene, people with no previous VR or gaming experience often preferred to just let the visuals surprise them. They used mainly head movement to change their point of view, whereas people with previous VR or gaming experience quickly learned inherent capabilities of the system. The AudioVirtualizer system is most suited to respond to electronic soundscapes and microphone input.
Copyright © 2021 Adinda van ’t Klooster et al., licensed to EAI. This is an open access article distributed under the terms of the Creative Commons Attribution license, which permits unlimited use, distribution and reproduction in any medium so long as the original work is properly cited.
