2nd International Conference on INtelligent TEchnologies for interactive enterTAINment

Research Article

Beyond the Beat: Modelling Intentions in a Virtual Conductor

Download487 downloads
  • @INPROCEEDINGS{10.4108/ICST.INTETAIN2008.2489,
        author={Mark ter Maat and Rob M. Ebbers and Dennis Reidsma and Anton Nijholt},
        title={Beyond the Beat: Modelling Intentions in a Virtual Conductor},
        proceedings={2nd International Conference on  INtelligent TEchnologies for interactive enterTAINment},
        publisher={ICST},
        proceedings_a={INTETAIN},
        year={2010},
        month={5},
        keywords={Intelligent agents virtual humans audio analysis entertainment computing},
        doi={10.4108/ICST.INTETAIN2008.2489}
    }
    
  • Mark ter Maat
    Rob M. Ebbers
    Dennis Reidsma
    Anton Nijholt
    Year: 2010
    Beyond the Beat: Modelling Intentions in a Virtual Conductor
    INTETAIN
    ICST
    DOI: 10.4108/ICST.INTETAIN2008.2489
Mark ter Maat1,*, Rob M. Ebbers1,*, Dennis Reidsma1,*, Anton Nijholt1,*
  • 1: Human Media Interaction, University of Twente, PO Box 217, 7500 AE Enschede, the Netherlands. +31 53 489 3740
*Contact email: mark.termaat@gmail.com, rob.ebbers@gmail.com, dennisr@ewi.utwente.nl, anijholt@ewi.utwente.nl

Abstract

We describe our research on designing and implementing a Virtual Conductor. That is, a virtual human (embodied agent) that acts like a human conductor in its interaction with a real, human orchestra. We reported previously on a first version that used a digital musical score to lead an orchestra. This conductor was able to conduct the beat with a certain tempo and dynamics, and to correct the tempo if necessary, using advanced audio analysis. We observed this Virtual Conductor at work during various performances for which he was invited. These performances made us aware of shortcomings. Therefore we took a closer look at the interaction between conductors and musicians in practice, both during performances and during rehearsals, and based on this study we introduced conducting gestures that display the intentions of a conductor and developed rehearsal modules. Apart from the literature on conducting we took into account videos of human conductors and interviewed human conductors. In addition we introduced principles from conversational analysis in the new design of our Virtual Conductor.