Intelligent Technologies for Interactive Entertainment. 5th International ICST Conference, INTETAIN 2013, Mons, Belgium, July 3-5, 2013, Revised Selected Papers

Research Article

MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters

Download138 downloads
  • @INPROCEEDINGS{10.1007/978-3-319-03892-6_21,
        author={Nicolas d’Alessandro and Maria Astrinaki and Thierry Dutoit},
        title={MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters},
        proceedings={Intelligent Technologies for Interactive Entertainment. 5th International ICST Conference, INTETAIN 2013, Mons, Belgium, July 3-5, 2013, Revised Selected Papers},
        proceedings_a={INTETAIN},
        year={2014},
        month={6},
        keywords={speech synthesis software library performative media streaming architecture HTS MAGE realtime audio software face tracking mapping},
        doi={10.1007/978-3-319-03892-6_21}
    }
    
  • Nicolas d’Alessandro
    Maria Astrinaki
    Thierry Dutoit
    Year: 2014
    MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters
    INTETAIN
    Springer
    DOI: 10.1007/978-3-319-03892-6_21
Nicolas d’Alessandro1,*, Maria Astrinaki1,*, Thierry Dutoit1,*
  • 1: University of Mons
*Contact email: nda@numediart.org, maria.astrinaki@umons.ac.be, thierry.dutoit@umons.ac.be

Abstract

In this paper, we illustrate the use of the MAGE performative speech synthesizer through its application to the conversion of realtime-measured facial features with FaceOSC into speech synthesis features such as vocal tract shape or intonation. MAGE is a new software library for using HMM-based speech synthesis in reactive programming environments. MAGE uses a rewritten version of the HTS engine enabling the computation of speech audio samples on a two-label window instead of the whole sentence. Only this feature enables the realtime mapping of facial attributes to synthesis parameters.