SPE-38.1
DISTILHUBERT: SPEECH REPRESENTATION LEARNING BY LAYER-WISE DISTILLATION OF HIDDEN-UNIT BERT
Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee, National Taiwan University, Taiwan
Session:
Self Supervised Learning for Speech Recognition
Track:
Speech and Language Processing
Location:
Gather Area C
Presentation Time:
Tue, 10 May, 21:00 - 21:45 China Time (UTC +8)
Tue, 10 May, 13:00 - 13:45 UTC
Tue, 10 May, 13:00 - 13:45 UTC
Session Chair:
Hung-Yi Lee, National Taiwan University
Session SPE-38
SPE-38.1: DISTILHUBERT: SPEECH REPRESENTATION LEARNING BY LAYER-WISE DISTILLATION OF HIDDEN-UNIT BERT
Heng-Jui Chang, Shu-wen Yang, Hung-yi Lee, National Taiwan University, Taiwan
SPE-38.2: IMPROVING SELF-SUPERVISED LEARNING FOR SPEECH RECOGNITION WITH INTERMEDIATE LAYER SUPERVISION
Chengyi Wang, Zhenglu Yang, Nankai University, China; Yu Wu, Sanyuan Chen, Shujie Liu, Jinyu Li, Yao Qian, Microsoft, China
SPE-38.3: WAV2VEC-SWITCH: CONTRASTIVE LEARNING FROM ORIGINAL-NOISY SPEECH PAIRS FOR ROBUST SPEECH RECOGNITION
Yiming Wang, Jinyu Li, Yao Qian, Chengyi Wang, Yu Wu, Microsoft Corporation, United States of America; Heming Wang, The Ohio State University, United States of America
SPE-38.4: EFFICIENT ADAPTER TRANSFER OF SELF-SUPERVISED SPEECH MODELS FOR AUTOMATIC SPEECH RECOGNITION
Bethan Thomas, Salah Karout, Huawei R&D UK, United Kingdom of Great Britain and Northern Ireland; Samuel Kessler, University of Oxford, United Kingdom of Great Britain and Northern Ireland
SPE-38.5: AN EXPLORATION OF HUBERT WITH LARGE NUMBER OF CLUSTER UNITS AND MODEL ASSESSMENT USING BAYESIAN INFORMATION CRITERION
Takashi Maekaku, Yuya Fujita, Yahoo Japan Corporation, Japan; Xuankai Chang, Shinji Watanabe, Carnegie Mellon University, United States of America
SPE-38.6: OPTIMIZE WAV2VEC2S ARCHITECTURE FOR SMALL TRAINING SET THROUGH ANALYZING ITS PRE-TRAINED MODELS ATTENTION PATTERN
Liu Chen, Meysam Asgari, Hiroko Dodge, Oregon Health & Science University, United States of America