| Paper ID | AUD-1.1 |
| Paper Title |
RETHINKING THE SEPARATION LAYERS IN SPEECH SEPARATION NETWORKS |
| Authors |
Yi Luo, Columbia University, United States; Zhuo Chen, Microsoft Corporation, United States; Cong Han, Columbia University, United States; Chenda Li, Shanghai Jiao Tong University, China; Tianyan Zhou, Microsoft Corporation, United States; Nima Mesgarani, Columbia University, United States |
| Session | AUD-1: Audio and Speech Source Separation 1: Speech Separation |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 13:00 - 13:45 |
| Presentation Time: | Tuesday, 08 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Audio and Acoustic Signal Processing: [AUD-SEP] Audio and Speech Source Separation |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Modules in all existing speech separation networks can be categorized into single-input-multi-output (SIMO) modules and single-input-single-output (SISO) modules. SIMO modules generate more outputs than input, and SISO modules keep the numbers of input and output the same. While the majority of separation models only contain SIMO architectures, it has also been shown that certain two-stage separation systems integrated with a post-enhancement SISO module can improve the separation quality. Why performance improvements can be achieved by incorporating the SISO modules? Are SIMO modules always necessary? In this paper, we empirically examine those questions by designing models with varying configurations in the SIMO and SISO modules. We show that comparing with the standard SIMO-only design, a mixed SIMO-SISO design with a same model size is able to improve the separation performance especially under low-overlap conditions. We further validate the necessity of SIMO modules and show that SISO-only models are still able to perform separation without sacrificing the performance. The observations allow us to rethink the model design paradigm and present different views on how the separation is performed. |
