| Paper ID | AUD-3.3 |
| Paper Title |
AUTOMATIC MULTITRACK MIXING WITH A DIFFERENTIABLE MIXING CONSOLE OF NEURAL AUDIO EFFECTS |
| Authors |
Christian J. Steinmetz, Dolby Laboratories and Universitat Pomepu Fabra, Spain; Jordi Pons, Santiago Pascual, Joan Serrà, Dolby Laboratories, Spain |
| Session | AUD-3: Music Signal Analysis, Processing, and Synthesis 1: Deep Learning |
| Location | Gather.Town |
| Session Time: | Tuesday, 08 June, 14:00 - 14:45 |
| Presentation Time: | Tuesday, 08 June, 14:00 - 14:45 |
| Presentation |
Poster
|
| Topic |
Audio and Acoustic Signal Processing: [AUD-MSP] Music Signal Analysis, Processing and Synthesis |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Applications of deep learning to automatic multitrack mixing are largely unexplored. This is partly due to the limited available data, coupled with the fact that such data is relatively unstructured and variable. To address these challenges, we propose a domain-inspired model with a strong inductive bias for the mixing task. We achieve this with the application of pre-trained sub-networks and weight sharing, as well as with a sum/difference stereo loss function. The proposed model can be trained with a limited number of examples, is permutation invariant with respect to the input ordering, and places no limit on the number of input sources. Furthermore, it produces human-readable mixing parameters, allowing users to manually adjust or refine the generated mix. Results from a perceptual evaluation involving audio engineers indicate that our approach generates mixes that outperform baseline approaches. To the best of our knowledge, this work demonstrates the first approach in learning multitrack mixing conventions from real-world data at the waveform level, without knowledge of the underlying mixing parameters. |
