Detecting Information Relays in Deep Neural Networks
원문 초록
Deep learning of artificial neural networks (ANNs) is creating highly functional processes that are, unfortunately, nearly as hard to interpret as their biological counterparts. Identification of functional modules in natural brains plays an important role in cognitive and neuroscience alike, and can be carried out using a wide range of technologies such as fMRI, EEG/ERP, MEG, or calcium imaging. However, we do not have such robust methods at our disposal when it comes to understanding functional modules in artificial neural networks. Ideally, understanding which parts of an artificial neural network perform what function might help us to address a number of vexing problems in ANN research, such as catastrophic forgetting and overfitting. Furthermore, revealing a network's modularity could improve our trust in them by making these black boxes more transparent. Here, we introduce a new information-theoretic concept that proves useful in understanding and analyzing a network's functional modularity: the relay information $I_R$. The relay information measures how much information groups of neurons that participate in a particular function (modules) relay from inputs to outputs. Combined with a greedy search algorithm, relay information can be used to identify computational modules in neural networks. We also show that the functionality of modules correlates with the amount of relay information they carry.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Insights into Schizophrenia: Leveraging Machine Learning for Early Identification via EEG, ERP, and Demographic Attributes
Dataset combining EEG, eye-tracking, and high-speed video for ocular activity analysis across BCI paradigms
A Statistical Approach for Synthetic EEG Data Generation
Decoding Covert Speech from EEG Using a Functional Areas Spatio-Temporal Transformer
EEG-CLIP : Learning EEG representations from natural language descriptions
Spatial Distillation based Distribution Alignment (SDDA) for Cross-Headset EEG Classification
토론
이 논문과 연결된 포럼 글이 아직 없습니다.