| Paper ID | BIO-11.3 |
| Paper Title |
Decoding neural representations of rhythmic sounds from magnetoencephalography |
| Authors |
Pei-Chun Chang, Jia-Ren Chang, Po-Yu Chen, Li-Kai Cheng, Jen-Chuen Hsieh, National Yang Ming Chiao Tung University, Taiwan; Hsin-Yen Yu, Taipei National University of the Arts, Taiwan; Li-Fen Chen, Yong-Sheng Chen, National Yang Ming Chiao Tung University, Taiwan |
| Session | BIO-11: Deep Learning for Physiological Signals |
| Location | Gather.Town |
| Session Time: | Thursday, 10 June, 13:00 - 13:45 |
| Presentation Time: | Thursday, 10 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Biomedical Imaging and Signal Processing: [BIO] Biomedical signal processing |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Neuroscience studies have revealed neural processes involving rhythm perception, suggesting that brain encodes rhythmic sounds and embeds information in neural activity. In this work, we investigate how to extract rhythmic information embedded in the brain responses and to decode the original audio waveforms from the extracted information. A spatiotemporal convolutional neural network is adopted to extract compact rhythm-related representations from the noninvasively measured magnetoencephalographic (MEG) signals evoked by listening to rhythmic sounds. These learned MEG representations are then used to condition an audio generator network for the synthesis of the original rhythmic sounds. In the experiments, we evaluated the proposed method by using the MEG signals recorded from eight participants and demonstrated that the generated rhythms are highly related to those evoking the MEG signals. Interestingly, we found that the auditory-related MEG channels reveal high importance in en- coding rhythmic representations, the distribution of these representations relate to the timing of beats, and the behavior performance is consistent with the performance of neural decoding. These results suggest that the proposed method can synthesize rhythms by decoding neural representations from MEG. |
