MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters

Nicolas d’Alessandro; Maria Astrinaki; Thierry Dutoit

Intelligent Technologies for Interactive Entertainment. 5th International ICST Conference, INTETAIN 2013, Mons, Belgium, July 3-5, 2013, Revised Selected Papers

Research Article

MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters

Download

541 downloads

Cite: BibTeX Plain Text

@INPROCEEDINGS{10.1007/978-3-319-03892-6_21,
    author={Nicolas d’Alessandro and Maria Astrinaki and Thierry Dutoit},
    title={MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters},
    proceedings={Intelligent Technologies for Interactive Entertainment. 5th International ICST Conference, INTETAIN 2013, Mons, Belgium, July 3-5, 2013, Revised Selected Papers},
    proceedings_a={INTETAIN},
    year={2014},
    month={6},
    keywords={speech synthesis software library performative media streaming architecture HTS MAGE realtime audio software face tracking mapping},
    doi={10.1007/978-3-319-03892-6_21}
}

Nicolas d’Alessandro
Maria Astrinaki
Thierry Dutoit
Year: 2014
MAGEFACE: Performative Conversion of Facial Characteristics into Speech Synthesis Parameters
INTETAIN
Springer
DOI: 10.1007/978-3-319-03892-6_21

Nicolas d’Alessandro¹^,*, Maria Astrinaki¹^,*, Thierry Dutoit¹^,*

1: University of Mons

*Contact email: nda@numediart.org, maria.astrinaki@umons.ac.be, thierry.dutoit@umons.ac.be

Abstract

In this paper, we illustrate the use of the MAGE performative speech synthesizer through its application to the conversion of realtime-measured facial features with FaceOSC into speech synthesis features such as vocal tract shape or intonation. MAGE is a new software library for using HMM-based speech synthesis in reactive programming environments. MAGE uses a rewritten version of the HTS engine enabling the computation of speech audio samples on a two-label window instead of the whole sentence. Only this feature enables the realtime mapping of facial attributes to synthesis parameters.

Keywords: speech synthesis software library performative media streaming architecture HTS MAGE realtime audio software face tracking mapping

Published: 2014-06-19

: http://dx.doi.org/10.1007/978-3-319-03892-6_21