A Direct Speech-to-Speech Neural Network Methodology for Spanish-English Translation

In this work, a novel direct speech-to-speech methodology for translation is proposed; it is based on an LSTM neural network structure which has proven useful for translation in the classical way, i.e., the one consisting of three stages: speech-to-text conversion, text-to-text translation, and text-to-speech synthesis. In contrast with traditional approaches, the one in this work belongs to the recently appeared idea of direct translation without text representation, as this sort of training better corresponds to the way oral language learning takes place in humans. As a proof of concept digits are translated from an audio source in Spanish and pronounced as an audio signal in English. Advantages and disadvantages of the proposal when compared with traditional methodologies are discussed.