SPE-87.4
KNOWLEDGE DISTILLATION FOR NEURAL TRANSDUCERS FROM LARGE SELF-SUPERVISED PRE-TRAINED MODELS
Xiaoyu Yang, Qiujia Li, Phil Woodland, University of Cambridge, United Kingdom of Great Britain and Northern Ireland
Session:
Speech Recognition: Knowledge Transfer and Contextualization
Track:
Speech and Language Processing
Location:
Gather Area C
Presentation Time:
Fri, 13 May, 22:00 - 22:45 China Time (UTC +8)
Fri, 13 May, 14:00 - 14:45 UTC
Fri, 13 May, 14:00 - 14:45 UTC
Session Chair:
Duc Le, Meta
Session SPE-87
SPE-87.1: KNOWLEDGE TRANSFER FROM LARGE-SCALE PRETRAINED LANGUAGE MODELS TO END-TO-END SPEECH RECOGNIZERS
Yotaro Kubo, Shigeki Karita, Michiel Bacchiani, Google, Japan
SPE-87.2: IMPROVING CTC-BASED SPEECH RECOGNITION VIA KNOWLEDGE TRANSFERRING FROM PRE-TRAINED LANGUAGE MODELS
Keqi Deng, Gaofeng Cheng, Ji Xu, Pengyuan Zhang, Institute of Acoustics, Chinese Academy of Sciences, China; Songjun Cao, Yike Zhang, Long Ma, Tencent Cloud Xiaowei, China
SPE-87.3: IMPROVING NON-AUTOREGRESSIVE END-TO-END SPEECH RECOGNITION WITH PRE-TRAINED ACOUSTIC AND LANGUAGE MODELS
Keqi Deng, Zehui Yang, Gaofeng Cheng, Pengyuan Zhang, Institute of Acoustics, Chinese Academy of Sciences, China; Shinji Watanabe, Carnegie Mellon University, United States of America; Yosuke Higuchi, Waseda University, Japan
SPE-87.4: KNOWLEDGE DISTILLATION FOR NEURAL TRANSDUCERS FROM LARGE SELF-SUPERVISED PRE-TRAINED MODELS
Xiaoyu Yang, Qiujia Li, Phil Woodland, University of Cambridge, United Kingdom of Great Britain and Northern Ireland
SPE-87.5: IMPROVING END-TO-END CONTEXTUAL SPEECH RECOGNITION WITH FINE-GRAINED CONTEXTUAL KNOWLEDGE SELECTION
Minglun Han, Shiyu Zhou, Bo Xu, Institute of Automation, Chinese Academy of Sciences, China; Linhao Dong, Zhenlin Liang, Meng Cai, Zejun Ma, Bytedance AI Lab, China
SPE-87.6: CONTEXTUAL ADAPTERS FOR PERSONALIZED SPEECH RECOGNITION IN NEURAL TRANSDUCERS
Kanthashree Mysore Sathyendra, Thejaswi Muniyappa, Feng-Ju Chang, Jing Liu, Jinru Su, Grant P. Strimel, Athanasios Mouchtaris, Siegfried Kunzmann, Amazon, United States of America