EIT-1M: One Million EEG-Image-Text Pairs for Human Visual-textual Recognition and More
원문 초록
Recently, electroencephalography (EEG) signals have been actively incorporated to decode brain activity to visual or textual stimuli and achieve object recognition in multi-modal AI. Accordingly, endeavors have been focused on building EEG-based datasets from visual or textual single-modal stimuli. However, these datasets offer limited EEG epochs per category, and the complex semantics of stimuli presented to participants compromise their quality and fidelity in capturing precise brain activity. The study in neuroscience unveils that the relationship between visual and textual stimulus in EEG recordings provides valuable insights into the brain's ability to process and integrate multi-modal information simultaneously. Inspired by this, we propose a novel large-scale multi-modal dataset, named EIT-1M, with over 1 million EEG-image-text pairs. Our dataset is superior in its capacity of reflecting brain activities in simultaneously processing multi-modal information. To achieve this, we collected data pairs while participants viewed alternating sequences of visual-textual stimuli from 60K natural images and category-specific texts. Common semantic categories are also included to elicit better reactions from participants' brains. Meanwhile, response-based stimulus timing and repetition across blocks and sessions are included to ensure data diversity. To verify the effectiveness of EIT-1M, we provide an in-depth analysis of EEG data captured from multi-modal stimuli across different categories and participants, along with data quality scores for transparency. We demonstrate its validity on two tasks: 1) EEG recognition from visual or textual stimuli or both and 2) EEG-to-visual generation.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
3D 시각 지각을 위한 이중 스트림 EEG 디코oding
Dual-Stream EEG Decoding for 3D Visual Perception
본 논문은 생물학적 시각을 모방한 이중 경로(dual pathway) 아키텍처를 통해 3D 형상(shape) 지각을 위한 새로운 뇌 디코딩 모델을 탐구한다. 우리의 생체 모방(bio-inspired) 접근법은 연속 회전 중 ventral pathway와 dorsal pathway에서 영감을 받아 물체 정체성(object identity)과 공간 방향(spatial orientation)을 위한 별도의 디코딩 모듈을 구현한다. 각도 예측에는 원형 회귀(circular regression)를 사용하고, 3D 재구성을 위해 EEG 조건부 다중 시점(multiview) diffusion을 개발한다. 우리 접근법은 EEG 신호에서 물체 정체성과 공간 방향을 모두 성공적으로 디코딩하고 신경 활동으로부터 3D 재구성을 가능하게 하며, 해석 가능성 분석은 물체 및 각도 디코딩을 지원하는 과정에서 정적인 ventral 우세가 아니라 ventral, dorsal, 운동(motor) 관련 채널의 시간적으로 구조화된 관여를 보여준다.
ViEEG: Hierarchical Neural Coding with Cross-Modal Progressive Enhancement for EEG-Based Visual Decoding
eegFloss: A Python package for refining sleep EEG recordings using machine learning models
DBConformer: Dual-Branch Convolutional Transformer for EEG Decoding
AGTCNet: A Graph-Temporal Approach for Principled Motor Imagery EEG Classification
Cross-Modal Epileptic Signal Harmonization: Frequency Domain Mapping Quantization for Pre-training a Unified Neurophysiological Transformer
토론
이 논문과 연결된 포럼 글이 아직 없습니다.