2021 IEEE International Conference on Acoustics, Speech and Signal Processing
6-11 June 2021 • Toronto, Ontario, Canada
Extracting Knowledge from Information
| Paper ID | SPE-39.4 | ||
| Paper Title | EAT: ENHANCED ASR-TTS FOR SELF-SUPERVISED SPEECH RECOGNITION | ||
| Authors | Murali Karthick Baskar, Lukáš Burget, Brno University of Technology, Czechia; Shinji Watanabe, Johns Hopkins University, United States; Ramon Astudillo, IBM T. J. Watson Research Center, United States; Jan "Honza" Cernocky, Brno University of Technology, Czechia | ||
| Session | SPE-39: Speech Recognition 13: Acoustic Modeling 1 | ||
| Location | Gather.Town | ||
| Session Time: | Thursday, 10 June, 15:30 - 16:15 | ||
| Presentation Time: | Thursday, 10 June, 15:30 - 16:15 | ||
| Presentation | Poster | ||
| Topic | Speech Processing: [SPE-RECO] Acoustic Modeling for Automatic Speech Recognition | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | Self-supervised ASR-TTS models suffer in out-of-domain data conditions. Here we propose an enhanced ASR-TTS (EAT) model that incorporates two main features: 1) The ASR$\rightarrow$TTS direction is equipped with a language model reward to penalize the ASR hypotheses before forwarding it to TTS. 2) In the TTS$\rightarrow$ASR direction, a hyper-parameter is introduced to scale the attention context from synthesized speech before sending it to ASR to handle out-of-domain data. Training strategies and the effectiveness of the EAT model are explored under out-of-domain data conditions. The results show that EAT reduces the performance gap between supervised and self-supervised training significantly by absolute 2.6\% and 2.7\% on Librispeech and BABEL respectively. | ||