Relating the fundamental frequency of speech with EEG using a dilated convolutional network
원문 초록
To investigate how speech is processed in the brain, we can model the relation between features of a natural speech signal and the corresponding recorded electroencephalogram (EEG). Usually, linear models are used in regression tasks. Either EEG is predicted, or speech is reconstructed, and the correlation between predicted and actual signal is used to measure the brain's decoding ability. However, given the nonlinear nature of the brain, the modeling ability of linear models is limited. Recent studies introduced nonlinear models to relate the speech envelope to EEG. We set out to include other features of speech that are not coded in the envelope, notably the fundamental frequency of the voice (f0). F0 is a higher-frequency feature primarily coded at the brainstem to midbrain level. We present a dilated-convolutional model to provide evidence of neural tracking of the f0. We show that a combination of f0 and the speech envelope improves the performance of a state-of-the-art envelope-based model. This suggests the dilated-convolutional model can extract non-redundant information from both f0 and the envelope. We also show the ability of the dilated-convolutional model to generalize to subjects not included during training. This latter finding will accelerate f0-based hearing diagnosis.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Dataset combining EEG, eye-tracking, and high-speed video for ocular activity analysis across BCI paradigms
A Statistical Approach for Synthetic EEG Data Generation
Decoding Covert Speech from EEG Using a Functional Areas Spatio-Temporal Transformer
EEG-CLIP : Learning EEG representations from natural language descriptions
Spatial Distillation based Distribution Alignment (SDDA) for Cross-Headset EEG Classification
Toward Foundational Model for Sleep Analysis Using a Multimodal Hybrid Self-Supervised Learning Framework
토론
이 논문과 연결된 포럼 글이 아직 없습니다.