Uncovering the fundamental neural correlates of biological intelligence, developing mathematical models, and conducting computational simulations are critical for advancing new paradigms in artificial intelligence (AI). In this study, we implemented a comprehensive visual decision-making model that spans from visual input to behavioral output, using a neural dynamics modeling approach. Drawing inspiration from the key components of the dorsal visual pathway in primates, our model not only aligns closely with human behavior but also reflects neural activities in primates, and achieving accuracy comparable to convolutional neural networks (CNNs). Moreover, magnetic resonance imaging (MRI) identified key neuroimaging features such as structural connections and functional connectivity that are associated with performance in perceptual decision-making tasks. A neuroimaging-informed fine-tuning approach was introduced and applied to the model, leading to performance improvements that paralleled the behavioral variations observed among subjects. Compared to classical deep learning models, our model more accurately replicates the behavioral performance of biological intelligence, relying on the structural characteristics of biological neural networks rather than extensive training data, and demonstrating enhanced resilience to perturbation.
Neuroimaging studies, such as the Human Connectome Project (HCP), often collect multi-faceted and multi-block data to study the complex human brain. However, these data are often analyzed in a pairwise fashion, which can hinder our understanding of how different brain-related measures interact with each other. In this study, we comprehensively analyze the multi-block HCP data using the Data Integration via Analysis of Subspaces (DIVAS) method. We integrate structural and functional brain connectivity, substance use, cognition, and genetics in an exhaustive five-block analysis. This gives rise to the important finding that genetics is the single data modality most predictive of brain connectivity, outside of brain connectivity itself. Nearly 14\% of the variation in functional connectivity (FC) and roughly 12\% of the variation in structural connectivity (SC) is attributed to shared spaces with genetics. Moreover, investigations of shared space loadings provide interpretable associations between particular brain regions and drivers of variability, such as alcohol consumption in the substance-use data block. Novel Jackstraw hypothesis tests are developed for the DIVAS framework to establish statistically significant loadings. For example, in the (FC, SC, and Substance Use) shared space, these novel hypothesis tests highlight largely negative functional and structural connections suggesting the brain's role in physiological responses to increased substance use. Furthermore, our findings have been validated using a subset of genetically relevant siblings or twins not studied in the main analysis.
We propose a novel approach to investigate the brain mechanisms that support coordination of behavior between individuals. Brain states in single individuals defined by the patterns of functional connectivity between brain regions are used to create joint symbolic representations of the evolution of brain states in two or more individuals performing a task together. These symbolic dynamics can be analyzed to reveal aspects of the dynamics of joint brain states that are related to coordination or other interactive behaviors. We apply this approach to simultaneous electroencephalographic (EEG) data from pairs of subjects engaged in two different modes of finger-tapping coordination tasks (synchronization and syncopation) under different interaction conditions (Uncoupled, Leader-Follower, and Mutual) to explore the neural mechanisms of multi-person motor coordination. Our results reveal that the dyads exhibit mostly the same joint symbols in different interaction conditions - the most important differences are reflected in the symbolic dynamics. Recurrence analysis shows that interaction influences the dwell time in specific joint symbols and the structure of joint symbol sequences (motif length). In synchronization, increasing feedback promotes stability with longer dwell times and motif length. In syncopation, Leader-Follower interactions enhance stability (increase dwell time and motif length), but Mutual feedback dramatically reduces stability. Network analysis reveals distinct topological changes with task and feedback. In synchronization, stronger coupling stabilizes a few states restricting the pattern of flow between states, preserving a core-periphery structure of the joint brain states. In syncopation, a more distributed flow amongst a larger set of joint brain states reduces the dominance of core joint brain states.
Cognition refers to the function of information perception and processing, which is the fundamental psychological essence of human beings. It is responsible for reasoning and decision-making, while its evaluation is significant for the aviation domain in mitigating potential safety risks. Existing studies tend to use varied methods for cognitive state evaluation yet have limitations in timeliness, generalisation, and interpretability. Accordingly, this study adopts brain functional connectivity with electroencephalography signals to capture associations in brain regions across multiple subjects for evaluating real-time cognitive states. Specifically, a virtual reality-based flight platform is constructed with multi-screen embedded. Three distinctive cognitive tasks are designed and each has three degrees of difficulty. Thirty subjects are acquired for analysis and evaluation. The results are interpreted through different perspectives, including inner-subject and cross-subject for task-wise and gender-wise underlying brain functional connectivity. Additionally, this study incorporates questionnaire-based, task performance-based, and physiological measure-based approaches to fairly label the trials. A multi-class cognitive state evaluation is further conducted with the active brain connections. Benchmarking results demonstrate that the identified brain regions have considerable influences in cognition, with a multi-class accuracy rate of 95.83% surpassing existing studies. The derived findings bring significance to understanding the dynamic relationships among human brain functional regions, cross-subject cognitive behaviours, and decision-making, which have promising practical application values.
Multimodal neuroimaging modeling has becomes a widely used approach but confronts considerable challenges due to heterogeneity, which encompasses variability in data types, scales, and formats across modalities. This variability necessitates the deployment of advanced computational methods to integrate and interpret these diverse datasets within a cohesive analytical framework. In our research, we amalgamate functional magnetic resonance imaging, diffusion tensor imaging, and structural MRI into a cohesive framework. This integration capitalizes on the unique strengths of each modality and their inherent interconnections, aiming for a comprehensive understanding of the brain's connectivity and anatomical characteristics. Utilizing the Glasser atlas for parcellation, we integrate imaging derived features from various modalities: functional connectivity from fMRI, structural connectivity from DTI, and anatomical features from sMRI within consistent regions. Our approach incorporates a masking strategy to differentially weight neural connections, thereby facilitating a holistic amalgamation of multimodal imaging data. This technique enhances interpretability at connectivity level, transcending traditional analyses centered on singular regional attributes. The model is applied to the Human Connectome Project's Development study to elucidate the associations between multimodal imaging and cognitive functions throughout youth. The analysis demonstrates improved predictive accuracy and uncovers crucial anatomical features and essential neural connections, deepening our understanding of brain structure and function.
Human hearing range significantly surpasses the typical neuronal spiking frequency. Yet, neurons with their modest frequency range not only efficiently receive and process multiple orders higher frequency signals, but also demonstrate remarkable stability and adaptability to frequency variations in brain functional connectivity. Ability to process signals beyond the limitations of the receiver temporal or frequency (bandwidth) resolution is highly desirable yet requires complex design architectures. Using the FitzHugh-Nagumo model we reveal the harmonic fractal transformation of frequency and bandwidth, which enables the Nyquist rate integer (for low frequencies) and sub-integer (for high frequencies) multiplication. We also demonstrate for the first time that noise shaping can be achieved in a simple RLC-circuit without a requirement of a delay line. The discovered effect presents a novel regeneration type - 4R: re-amplifying, re-shaping, re-timing, and re-modulating and due to the fractal nature of transformation offers a remarkable regenerative efficiency. The effect is a generalization of phase locking to non-periodic encoded signals. The discovered physical mechanism explains how using neuronal functionality one can receive and process signals over an ultra-wide band (below or higher the spiking neuronal range by multiple orders) and below the noise floor.
Alzheimer's Disease (AD) is a currently incurable neurodegeneartive disease. Accurately detecting AD, especially in the early stage, represents a high research priority. AD is characterized by progressive cognitive impairments that are related to alterations in brain functional connectivity (FC). Based on this association, many studies have been published over the decades using FC and machine learning to differentiate AD from healthy aging. The most recent development in this detection method highlights the use of graph neural network (GNN) as the brain functionality analysis. In this paper, we proposed a stack of spatio-temporal feature extraction and graph generation based AD classification model using resting state fMRI. The proposed multi-level generated connectome (MLC) based graph convolutional network (GCN) (MLC-GCN) contains a multi-graph generation block and a GCN prediction block. The multi-graph generation block consists of a hierarchy of spatio-temporal feature extraction layers for extracting spatio-temporal rsfMRI features at different depths and building the corresponding connectomes. The GCN prediction block takes the learned multi-level connectomes to build and optimize GCNs at each level and concatenates the learned graphical features as the final predicting features for AD classification. Through independent cohort validations, MLC-GCN shows better performance for differentiating MCI, AD, and normal aging than state-of-art GCN and rsfMRI based AD classifiers. The proposed MLC-GCN also showed high explainability in terms of learning clinically reasonable connectome node and connectivity features from two independent datasets. While we only tested MLC-GCN on AD, the basic rsfMRI-based multi-level learned GCN based outcome prediction strategy is valid for other diseases or clinical outcomes.
Accurate diagnosis of depression is crucial for timely implementation of optimal treatments, preventing complications and reducing the risk of suicide. Traditional methods rely on self-report questionnaires and clinical assessment, lacking objective biomarkers. Combining fMRI with artificial intelligence can enhance depression diagnosis by integrating neuroimaging indicators. However, the specificity of fMRI acquisition for depression often results in unbalanced and small datasets, challenging the sensitivity and accuracy of classification models. In this study, we propose the Spatio-Temporal Aggregation Network (STANet) for diagnosing depression by integrating CNN and RNN to capture both temporal and spatial features of brain activity. STANet comprises the following steps:(1) Aggregate spatio-temporal information via ICA. (2) Utilize multi-scale deep convolution to capture detailed features. (3) Balance data using the SMOTE to generate new samples for minority classes. (4) Employ the AFGRU classifier, which combines Fourier transformation with GRU, to capture long-term dependencies, with an adaptive weight assignment mechanism to enhance model generalization. The experimental results demonstrate that STANet achieves superior depression diagnostic performance with 82.38% accuracy and a 90.72% AUC. The STFA module enhances classification by capturing deeper features at multiple scales. The AFGRU classifier, with adaptive weights and stacked GRU, attains higher accuracy and AUC. SMOTE outperforms other oversampling methods. Additionally, spatio-temporal aggregated features achieve better performance compared to using only temporal or spatial features. STANet outperforms traditional or deep learning classifiers, and functional connectivity-based classifiers, as demonstrated by ten-fold cross-validation.
The segregation and integration of infant brain networks undergo tremendous changes due to the rapid development of brain function and organization. Traditional methods for estimating brain modularity usually rely on group-averaged functional connectivity (FC), often overlooking individual variability. To address this, we introduce a novel approach utilizing Bayesian modeling to analyze the dynamic development of functional modules in infants over time. This method retains inter-individual variability and, in comparison to conventional group averaging techniques, more effectively detects modules, taking into account the stationarity of module evolution. Furthermore, we explore gender differences in module development under awake and sleep conditions by assessing modular similarities. Our results show that female infants demonstrate more distinct modular structures between these two conditions, possibly implying relative quiet and restful sleep compared with male infants.
Brain-computer interfaces (BCIs) enable users to interact with the external world using brain activity. Despite their potential in neuroscience and industry, BCI performance remains inconsistent in noninvasive applications, often prioritizing algorithms that achieve high classification accuracies while masking the neural mechanisms driving that performance. In this study, we investigated the interpretability of features derived from brain network lateralization, benchmarking against widely used techniques like power spectrum density (PSD), common spatial pattern (CSP), and Riemannian geometry. We focused on the spatial distribution of the functional connectivity within and between hemispheres during motor imagery tasks, introducing network-based metrics such as integration and segregation. Evaluating these metrics across multiple EEG-based BCI datasets, our findings reveal that network lateralization offers neurophysiological plausible insights, characterized by stronger lateralization in sensorimotor and frontal areas contralateral to imagined movements. While these lateralization features did not outperform CSP and Riemannian geometry in terms of classification accuracy, they demonstrated competitive performance against PSD alone and provided biologically relevant interpretation. This study underscores the potential of brain network lateralization as a new feature to be integrated in motor imagery-based BCIs for enhancing the interpretability of noninvasive applications.
Background: Deep learning models have shown promise in diagnosing neurodevelopmental disorders (NDD) like ASD and ADHD. However, many models either use graph neural networks (GNN) to construct single-level brain functional networks (BFNs) or employ spatial convolution filtering for local information extraction from rs-fMRI data, often neglecting high-order features crucial for NDD classification. Methods: We introduce a Multi-view High-order Network (MHNet) to capture hierarchical and high-order features from multi-view BFNs derived from rs-fMRI data for NDD prediction. MHNet has two branches: the Euclidean Space Features Extraction (ESFE) module and the Non-Euclidean Space Features Extraction (Non-ESFE) module, followed by a Feature Fusion-based Classification (FFC) module for NDD identification. ESFE includes a Functional Connectivity Generation (FCG) module and a High-order Convolutional Neural Network (HCNN) module to extract local and high-order features from BFNs in Euclidean space. Non-ESFE comprises a Generic Internet-like Brain Hierarchical Network Generation (G-IBHN-G) module and a High-order Graph Neural Network (HGNN) module to capture topological and high-order features in non-Euclidean space. Results: Experiments on three public datasets show that MHNet outperforms state-of-the-art methods using both AAL1 and Brainnetome Atlas templates. Extensive ablation studies confirm the superiority of MHNet and the effectiveness of using multi-view fMRI information and high-order features. Our study also offers atlas options for constructing more sophisticated hierarchical networks and explains the association between key brain regions and NDD. Conclusion: MHNet leverages multi-view feature learning from both Euclidean and non-Euclidean spaces, incorporating high-order information from BFNs to enhance NDD classification performance.
Human brains exhibit highly organized multiscale neurophysiological dynamics. Understanding those dynamic changes and the neuronal networks involved is critical for understanding how the brain functions in health and disease. Functional Magnetic Resonance Imaging (fMRI) is a prevalent neuroimaging technique for studying these complex interactions. However, analyzing fMRI data poses several challenges. Furthermore, most approaches for analyzing Functional Connectivity (FC) still rely on preprocessing or conventional methods, often built upon oversimplified assumptions. On top of that, those approaches often ignore frequency-related information despite evidence showing that fMRI data contain rich information that spans multiple timescales. This study introduces a novel methodology, Multiscale Functional Connectivity (MFC), to analyze fMRI data by decomposing the fMRI into their intrinsic modes, allowing us to separate the neurophysiological activation patterns at multiple timescales while separating them from other interfering components. Additionally, the proposed approach accounts for the natural nonlinear and nonstationary nature of fMRI and the particularities of each individual in a data-driven way. We evaluated the performance of our proposed methodology using three fMRI experiments. Our results demonstrate that our novel approach effectively separates the fMRI data into different timescales while identifying highly reliable functional connectivity patterns across individuals. In addition, we further extended our knowledge of how the FC for these three experiments spans among different timescales.
The human brain is a complex, dynamic network, which is commonly studied using functional magnetic resonance imaging (fMRI) and modeled as network of Regions of interest (ROIs) for understanding various brain functions. Recent studies utilize deep learning approaches to learn the brain network representation based on functional connectivity (FC) profile, broadly falling into two main categories. The Fixed-FC approaches, utilizing the FC profile which represents the linear temporal relation within the brain network, are limited by failing to capture informative brain temporal dynamics. On the other hand, the Dynamic-FC approaches, modeling the evolving FC profile over time, often exhibit less satisfactory performance due to challenges in handling the inherent noisy nature of fMRI data. To address these challenges, we propose Brain Masked Auto-Encoder (BrainMAE) for learning representations directly from fMRI time-series data. Our approach incorporates two essential components: a region-aware graph attention mechanism designed to capture the relationships between different brain ROIs, and a novel self-supervised masked autoencoding framework for effective model pre-training. These components enable the model to capture rich temporal dynamics of brain activity while maintaining resilience to inherent noise in fMRI data. Our experiments demonstrate that BrainMAE consistently outperforms established baseline methods by significant margins in four distinct downstream tasks. Finally, leveraging the model's inherent interpretability, our analysis of model-generated representations reveals findings that resonate with ongoing research in the field of neuroscience.
Dynamic functional connectivity (DFC) analysis has been widely applied to functional magnetic resonance imaging (fMRI) data to reveal time-varying dynamic changes of brain states. The sliding window method is by far the most popular DFC analysis method due to its simplicity. However, the sliding window method comes with some assumptions, namely the typically approach uses a single window which captures dynamics only within a specific frequency range. In this study, we propose a generalized approach to dynamics via a multi-dimensional random convolution (RandCon) DFC method that is able to effectively capture time-varying DFC at arbitrary time scales by extracting different local features from fMRI time series using a number of multi-dimensional random convolution kernels without the need for learning kernel weights. Compared to a standard sliding window method, multiplication of temporal derivatives (MTD) and phase synchrony methods, RandCon with the smallest kernel size (3 time points) showed notable improvements in performance on simulated data, particularly in terms of DFC temporal and spatial estimation in very short window/kernel size under different noise levels. Results from real fMRI data indicated that RandCon was more sensitive to gender differences than competing methods. Furthermore, we show that the sliding window method can be considered a special case of the proposed multi-dimensional convolution framework. The proposed method is simple and efficient significantly broadens the scope of dynamic functional connectivity research and offer theoretical and practical potential.
In signal processing, exploring complex systems through network representations has become an area of growing interest. This study introduces the modularity graph, a new graph-based feature, to highlight the relationship across the graph communities. After showing an application to the random graph class known as Stochastic Block Model, we consider the brain functional connectivity network estimated from real EEG data. The modularity graph provides a quantitative framework for examining the interactions between neuron clusters within the brain's network. The modularity graph works alongside multiscale community detection algorithms, thereby enabling the identification of community structures at various scales. After introducing the modularity graph, we apply it to the brain functional connectivity network, estimated from publicly available EEG recordings of motor imagery experiments. Statistical analysis across multiple scales shows that the modularity graph differs for the distinct brain connectivity states associated with various motor imagery tasks. This work emphasizes the application of signal on graph processing techniques to understand brain behavior during specific cognitive tasks, leveraging the novel modularity graph to identify patterns of brain connectivity in different cognitive conditions. This approach sets the stage for further signal on graph analysis to devise brain network modularity, and to gain insights into the motor imagery mechanisms.
Voxel hierarchy on dynamic brain graphs is produced by k core percolation on functional dynamic amplitude correlation of resting-state fMRI. Directed graphs and their afferent/efferent capacities are produced by Markov modeling of the universal cover of undirected graphs simultaneously with the calculation of volume entropy. Positive and unsigned negative brain graphs were analyzed separately on sliding-window representation to underpin the visualization and quantitation of mental dynamic states with their transitions. Voxel hierarchy animation maps of positive graphs revealed abrupt changes in coreness k and kmaxcore, which we called mental state transitions. Afferent voxel capacities of the positive graphs also revealed transient modules composed of dominating voxels/independent components and their exchanges representing mental state transitions. Animation and quantification plots of voxel hierarchy and afferent capacity corroborated each other in underpinning mental state transitions and afferent module exchange on the positive directed functional connectivity graphs. We propose the use of spatiotemporal trajectories of voxels on positive dynamic graphs to construct hierarchical structures by k core percolation and quantified in- and out-flows of information of voxels by volume entropy/directed graphs to subserve diverse resting mental state transitions on resting-state fMRI graphs in normal human individuals.
The brain's complex functionality emerges from network interactions that go beyond dyadic connections, with higher-order interactions significantly contributing to this complexity. One method of capturing higher-order interactions is through traversing the brain network using random walks. The efficacy of these random walks depends on the defined mutual interactions between two brain entities. More precise capture of higher-order interactions enables a better reflection of the brain's intrinsic neurophysiological characteristics. One well-established neurophysiological concept is Homotopic Functional Connectivity (HoFC), which illustrates the synchronized spontaneous activity between corresponding regions in the brain's left and right hemispheres. We employ node2vec, a random walk node embedding approach, alongside resting-state fMRI from the Human Connectome Project (HCP) to obtain higher-order feature vectors. We assess the efficacy of different functional connectivity parameterizations using HoFC. The results indicates that the quality of capturing higher-order interactions largely depends on the statistical dependency measure between brain regions. Higher-order interactions defined by partial correlation, better reflects HoFC compare to other statistical associations. In this case of first-order interactions, tangent space embedding more effectively demonstrates HoFC. The findings validate HoFC and underscore the importance of functional connectivity construction method in capturing intrinsic characteristics of the human brain.
Major depressive disorder (MDD) is a common mental disorder that typically affects a person's mood, cognition, behavior, and physical health. Resting-state functional magnetic resonance imaging (rs-fMRI) data are widely used for computer-aided diagnosis of MDD. While multi-site fMRI data can provide more data for training reliable diagnostic models, significant cross-site data heterogeneity would result in poor model generalizability. Many domain adaptation methods are designed to reduce the distributional differences between sites to some extent, but usually ignore overfitting problem of the model on the source domain. Intuitively, target data augmentation can alleviate the overfitting problem by forcing the model to learn more generalized features and reduce the dependence on source domain data. In this work, we propose a new augmentation-based unsupervised cross-domain fMRI adaptation (AUFA) framework for automatic diagnosis of MDD. The AUFA consists of 1) a graph representation learning module for extracting rs-fMRI features with spatial attention, 2) a domain adaptation module for feature alignment between source and target data, 3) an augmentation-based self-optimization module for alleviating model overfitting on the source domain, and 4) a classification module. Experimental results on 1,089 subjects suggest that AUFA outperforms several state-of-the-art methods in MDD identification. Our approach not only reduces data heterogeneity between different sites, but also localizes disease-related functional connectivity abnormalities and provides interpretability for the model.
Resting-state functional magnetic resonance imaging (rs-fMRI) is a noninvasive technique pivotal for understanding human neural mechanisms of intricate cognitive processes. Most rs-fMRI studies compute a single static functional connectivity matrix across brain regions of interest, or dynamic functional connectivity matrices with a sliding window approach. These approaches are at risk of oversimplifying brain dynamics and lack proper consideration of the goal at hand. While deep learning has gained substantial popularity for modeling complex relational data, its application to uncovering the spatiotemporal dynamics of the brain is still limited. We propose a novel interpretable deep learning framework that learns goal-specific functional connectivity matrix directly from time series and employs a specialized graph neural network for the final classification. Our model, DSAM, leverages temporal causal convolutional networks to capture the temporal dynamics in both low- and high-level feature representations, a temporal attention unit to identify important time points, a self-attention unit to construct the goal-specific connectivity matrix, and a novel variant of graph neural network to capture the spatial dynamics for downstream classification. To validate our approach, we conducted experiments on the Human Connectome Project dataset with 1075 samples to build and interpret the model for the classification of sex group, and the Adolescent Brain Cognitive Development Dataset with 8520 samples for independent testing. Compared our proposed framework with other state-of-art models, results suggested this novel approach goes beyond the assumption of a fixed connectivity matrix and provides evidence of goal-specific brain connectivity patterns, which opens up the potential to gain deeper insights into how the human brain adapts its functional connectivity specific to the task at hand.
Objective: fMRI and derived measures such as functional connectivity (FC) have been used to predict brain age, general fluid intelligence, psychiatric disease status, and preclinical neurodegenerative disease. However, it is not always clear that all demographic confounds, such as age, sex, and race, have been removed from fMRI data. Additionally, many fMRI datasets are restricted to authorized researchers, making dissemination of these valuable data sources challenging. Methods: We create a variational autoencoder (VAE)-based model, DemoVAE, to decorrelate fMRI features from demographics and generate high-quality synthetic fMRI data based on user-supplied demographics. We train and validate our model using two large, widely used datasets, the Philadelphia Neurodevelopmental Cohort (PNC) and Bipolar and Schizophrenia Network for Intermediate Phenotypes (BSNIP). Results: We find that DemoVAE recapitulates group differences in fMRI data while capturing the full breadth of individual variations. Significantly, we also find that most clinical and computerized battery fields that are correlated with fMRI data are not correlated with DemoVAE latents. An exception are several fields related to schizophrenia medication and symptom severity. Conclusion: Our model generates fMRI data that captures the full distribution of FC better than traditional VAE or GAN models. We also find that most prediction using fMRI data is dependent on correlation with, and prediction of, demographics. Significance: Our DemoVAE model allows for generation of high quality synthetic data conditioned on subject demographics as well as the removal of the confounding effects of demographics. We identify that FC-based prediction tasks are highly influenced by demographic confounds.