FoME: A Foundation Model for EEG using Adaptive Temporal-Lateral Attention Scaling
원문 초록
Electroencephalography (EEG) is a vital tool to measure and record brain activity in neuroscience and clinical applications, yet its potential is constrained by signal heterogeneity, low signal-to-noise ratios, and limited labeled datasets. In this paper, we propose FoME (Foundation Model for EEG), a novel approach using adaptive temporal-lateral attention scaling to address above-mentioned challenges. FoME is pre-trained on a diverse 1.7TB dataset of scalp and intracranial EEG recordings, comprising 745M parameters trained for 1,096k steps. Our model introduces two key innovations: a time-frequency fusion embedding technique and an adaptive time-lateral attention scaling (ATLAS) mechanism. These components synergistically capture complex temporal and spectral EEG dynamics, enabling FoME to adapt to varying patterns across diverse data streams and facilitate robust multi-channel modeling. Evaluations across four downstream tasks demonstrate FoME's superior performance in classification and forecasting applications, consistently achieving state-of-the-art results. To conclude, FoME establishes a new paradigm for EEG analysis, offering a versatile foundation that advances brain-computer interfaces, clinical diagnostics, and cognitive research across neuroscience and related fields. Our code will be available at https://github.com/1061413241/FoME.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
eegFloss: A Python package for refining sleep EEG recordings using machine learning models
DBConformer: Dual-Branch Convolutional Transformer for EEG Decoding
AGTCNet: A Graph-Temporal Approach for Principled Motor Imagery EEG Classification
Cross-Modal Epileptic Signal Harmonization: Frequency Domain Mapping Quantization for Pre-training a Unified Neurophysiological Transformer
Efficient and faithful reconstruction of dynamical attractors using homogeneous differentiators
On using AI for EEG-based BCI applications: problems, current challenges and future trends
토론
이 논문과 연결된 포럼 글이 아직 없습니다.