Beyond the Beat: Modelling Intentions in a Virtual Conductor

We describe our research on designing and implementing a Virtual Conductor. That is, a virtual human (embodied agent) that acts like a human conductor in its interaction with a real, human orchestra. We reported previously on a first version that used a digital musical score to lead an orchestra. This conductor was able to conduct the beat with a certain tempo and dynamics, and to correct the tempo if necessary, using advanced audio analysis. We observed this Virtual Conductor at work during various performances for which he was invited. These performances made us aware of shortcomings. Therefore we took a closer look at the interaction between conductors and musicians in practice, both during performances and during rehearsals, and based on this study we introduced conducting gestures that display the intentions of a conductor and developed rehearsal modules. Apart from the literature on conducting we took into account videos of human conductors and interviewed human conductors. In addition we introduced principles from conversational analysis in the new design of our Virtual Conductor.