Detecting the salient parts of motor-imagery electroencephalogram (MI-EEG) signals can enhance the performance of the brain-computer interface (BCI) system and reduce the computational burden required for processing lengthy MI-EEG signals. In this paper, we propose an unsupervised method based on the self-attention mechanism to detect the salient intervals of MI-EEG signals automatically. Our suggested method can be used as a preprocessing step within any BCI algorithm to enhance its performance. The effectiveness of the suggested method is evaluated on the most widely used BCI algorithm, the common spatial pattern (CSP) algorithm, using dataset 2a from BCI competition IV. The results indicate that the proposed method can effectively prune MI-EEG signals and significantly enhance the performance of the CSP algorithm in terms of classification accuracy.
A calibration procedure is required in motor imagery-based brain-computer interface (MI-BCI) to tune the system for new users. This procedure is time-consuming and prevents na\"ive users from using the system immediately. Developing a subject-independent MI-BCI system to reduce the calibration phase is still challenging due to the subject-dependent characteristics of the MI signals. Many algorithms based on machine learning and deep learning have been developed to extract high-level features from the MI signals to improve the subject-to-subject generalization of a BCI system. However, these methods are based on supervised learning and extract features useful for discriminating various MI signals. Hence, these approaches cannot find the common underlying patterns in the MI signals and their generalization level is limited. This paper proposes a subject-independent MI-BCI based on a supervised autoencoder (SAE) to circumvent the calibration phase. The suggested framework is validated on dataset 2a from BCI competition IV. The simulation results show that our SISAE model outperforms the conventional and widely used BCI algorithms, common spatial and filter bank common spatial patterns, in terms of the mean Kappa value, in eight out of nine subjects.
Speech impairments due to cerebral lesions and degenerative disorders can be devastating. For humans with severe speech deficits, imagined speech in the brain-computer interface has been a promising hope for reconstructing the neural signals of speech production. However, studies in the EEG-based imagined speech domain still have some limitations due to high variability in spatial and temporal information and low signal-to-noise ratio. In this paper, we investigated the neural signals for two groups of native speakers with two tasks with different languages, English and Chinese. Our assumption was that English, a non-tonal and phonogram-based language, would have spectral differences in neural computation compared to Chinese, a tonal and ideogram-based language. The results showed the significant difference in the relative power spectral density between English and Chinese in specific frequency band groups. Also, the spatial evaluation of Chinese native speakers in the theta band was distinctive during the imagination task. Hence, this paper would suggest the key spectral and spatial information of word imagination with specialized language while decoding the neural signals of speech.
Brain-computer interface (BCI) is challenging to use in practice due to the inter/intra-subject variability of electroencephalography (EEG). The BCI system, in general, necessitates a calibration technique to obtain subject/session-specific data in order to tune the model each time the system is utilized. This issue is acknowledged as a key hindrance to BCI, and a new strategy based on domain generalization has recently evolved to address it. In light of this, we've concentrated on developing an EEG classification framework that can be applied directly to data from unknown domains (i.e. subjects), using only data acquired from separate subjects previously. For this purpose, in this paper, we proposed a framework that employs the open-set recognition technique as an auxiliary task to learn subject-specific style features from the source dataset while helping the shared feature extractor with mapping the features of the unseen target dataset as a new unseen domain. Our aim is to impose cross-instance style in-variance in the same domain and reduce the open space risk on the potential unseen subject in order to improve the generalization ability of the shared feature extractor. Our experiments showed that using the domain information as an auxiliary network increases the generalization performance.
In a self-paced motor-imagery brain-computer interface (MI-BCI), the onsets of the MI commands presented in a continuous electroencephalogram (EEG) signal are unknown. To detect these onsets, most self-paced approaches apply a window function on the continuous EEG signal and split it into long segments for further analysis. As a result, the system has a high latency. To reduce the system latency, we propose an algorithm based on the time series prediction concept and use the data of the previously received time samples to predict the upcoming time samples. Our predictor is an encoder-decoder (ED) network built with long short-term memory (LSTM) units. The onsets of the MI commands are detected shortly by comparing the incoming signal with the predicted signal. The proposed method is validated on dataset IVc from BCI competition III. The simulation results show that the proposed algorithm improves the average F1-score achieved by the winner of the competition by 26.7% for latencies shorter than one second.
Brain-computer interfaces (BCIs), transform neural signals in the brain into in-structions to control external devices. However, obtaining sufficient training data is difficult as well as limited. With the advent of advanced machine learning methods, the capability of brain-computer interfaces has been enhanced like never before, however, these methods require a large amount of data for training and thus require data augmentation of the limited data available. Here, we use spiking neural networks (SNN) as data generators. It is touted as the next-generation neu-ral network and is considered as one of the algorithms oriented to general artifi-cial intelligence because it borrows the neural information processing from bio-logical neurons. We use the SNN to generate neural spike information that is bio-interpretable and conforms to the intrinsic patterns in the original neural data. Ex-periments show that the model can directly synthesize new spike trains, which in turn improves the generalization ability of the BCI decoder. Both the input and output of the spiking neural model are spike information, which is a brain-inspired intelligence approach that can be better integrated with BCI in the future.
The processing and classification of electroencephalographic signals (EEG) are increasingly performed using deep learning frameworks, such as convolutional neural networks (CNNs), to generate abstract features from brain data, automatically paving the way for remarkable classification prowess. However, EEG patterns exhibit high variability across time and uncertainty due to noise. It is a significant problem to be addressed in P300-based Brain Computer Interface (BCI) for smart home interaction. It operates in a non-optimal natural environment where added noise is often present. In this work, we propose a sequential unification of temporal convolutional networks (TCNs) modified to EEG signals, LSTM cells, with a fuzzy neural block (FNB), which we called EEG-TCFNet. Fuzzy components may enable a higher tolerance to noisy conditions. We applied three different architectures comparing the effect of using block FNB to classify a P300 wave to build a BCI for smart home interaction with healthy and post-stroke individuals. Our results reported a maximum classification accuracy of 98.6% and 74.3% using the proposed method of EEG-TCFNet in subject-dependent strategy and subject-independent strategy, respectively. Overall, FNB usage in all three CNN topologies outperformed those without FNB. In addition, we compared the addition of FNB to other state-of-the-art methods and obtained higher classification accuracies on account of the integration with FNB. The remarkable performance of the proposed model, EEG-TCFNet, and the general integration of fuzzy units to other classifiers would pave the way for enhanced P300-based BCIs for smart home interaction within natural settings.
Practical determination of physical recovery after intense exercise is a challenging topic that must include mechanical aspects as well as cognitive ones because most of physical sport activities, as well as professional activities (including brain computer interface-operated systems), require good shape in both of them. This paper presents a new online handwritten database of 20 healthy subjects. The main goal was to study the influence of several physical exercise stimuli in different handwritten tasks and to evaluate the recovery after strenuous exercise. To this aim, they performed different handwritten tasks before and after physical exercise as well as other measurements such as metabolic and mechanical fatigue assessment. Experimental results showed that although a fast mechanical recovery happens and can be measured by lactate concentrations and mechanical fatigue, this is not the case when cognitive effort is required. Handwriting analysis revealed that statistical differences exist on handwriting performance even after lactate concentration and mechanical assessment recovery. Conclusions: This points out a necessity of more recovering time in sport and professional activities than those measured in classic ways.
We aim to help users communicate their intent to machines using flexible, adaptive interfaces that translate arbitrary user input into desired actions. In this work, we focus on assistive typing applications in which a user cannot operate a keyboard, but can instead supply other inputs, such as webcam images that capture eye gaze or neural activity measured by a brain implant. Standard methods train a model on a fixed dataset of user inputs, then deploy a static interface that does not learn from its mistakes; in part, because extracting an error signal from user behavior can be challenging. We investigate a simple idea that would enable such interfaces to improve over time, with minimal additional effort from the user: online learning from user feedback on the accuracy of the interface's actions. In the typing domain, we leverage backspaces as feedback that the interface did not perform the desired action. We propose an algorithm called x-to-text (X2T) that trains a predictive model of this feedback signal, and uses this model to fine-tune any existing, default interface for translating user input into actions that select words or characters. We evaluate X2T through a small-scale online user study with 12 participants who type sentences by gazing at their desired words, a large-scale observational study on handwriting samples from 60 users, and a pilot study with one participant using an electrocorticography-based brain-computer interface. The results show that X2T learns to outperform a non-adaptive default interface, stimulates user co-adaptation to the interface, personalizes the interface to individual users, and can leverage offline data collected from the default interface to improve its initial performance and accelerate online learning.
Stable and accurate electroencephalogram (EEG) signal acquisition is fundamental in non-invasive brain-computer interface (BCI) technology. Commonly used EEG acquisition system's hardware and software are usually closed-source. Its inability to flexible expansion and secondary development is a major obstacle to real-time BCI research. This paper presents the Beijing University of Posts and Telecommunications EEG Acquisition Tool System named BEATS. It implements a comprehensive system from hardware to software, composed of the analog front-end, microprocessor, and software platform. BEATS is capable of collecting 32-channel EEG signals at a guaranteed sampling rate of 4k Hz with wireless transmission. Compared to state-of-the-art systems used in many EEG fields, it displays a better sampling rate. Using techniques including direct memory access, first in first out, and timer, the precision and stability of the acquisition are ensured at the microsecond level. An evaluation is conducted during 24 hours of continuous acquisitions. The data loss is 0 packets and the average maximum delay is only 0.07 s/h. Moreover, as an open source system, BEATS provides detailed design files, and adopts a plug-in structure and easy-to-access materials, which makes it can be quickly reproduced. Schematics, source code, and other materials of BEATS are available at https://github.com/buptantEEG/BEATS.
Brain-computer interface systems aim to facilitate human-computer interactions in a great deal by direct translation of brain signals for computers. Recently, using many electrodes has caused better performance in these systems. However, increasing the number of recorded electrodes leads to additional time, hardware, and computational costs besides undesired complications of the recording process. Channel selection has been utilized to decrease data dimension and eliminate irrelevant channels while reducing the noise effects. Furthermore, the technique lowers the time and computational costs in real-time applications. We present a channel selection method, which combines a sequential search method with a genetic algorithm called Deep GA Fitness Formation (DGAFF). The proposed method accelerates the convergence of the genetic algorithm and increases the system's performance. The system evaluation is based on a lightweight deep neural network that automates the whole model training process. The proposed method outperforms other channel selection methods in classifying motor imagery on the utilized dataset.
Motor imagery (MI) is a common brain computer interface (BCI) paradigm. EEG is non-stationary with low signal-to-noise, classifying motor imagery tasks of the same participant from different EEG recording sessions is generally challenging, as EEG data distribution may vary tremendously among different acquisition sessions. Although it is intuitive to consider the cross-session MI classification as a domain adaptation problem, the rationale and feasible approach is not elucidated. In this paper, we propose a Siamese deep domain adaptation (SDDA) framework for cross-session MI classification based on mathematical models in domain adaptation theory. The proposed framework can be easily applied to most existing artificial neural networks without altering the network structure, which facilitates our method with great flexibility and transferability. In the proposed framework, domain invariants were firstly constructed jointly with channel normalization and Euclidean alignment. Then, embedding features from source and target domain were mapped into the Reproducing Kernel Hilbert Space (RKHS) and aligned accordingly. A cosine-based center loss was also integrated into the framework to improve the generalizability of the SDDA. The proposed framework was validated with two classic and popular convolutional neural networks from BCI research field (EEGNet and ConvNet) in two MI-EEG public datasets (BCI Competition IV IIA, IIB). Compared to the vanilla EEGNet and ConvNet, the proposed SDDA framework was able to boost the MI classification accuracy by 15.2%, 10.2% respectively in IIA dataset, and 5.5%, 4.2% in IIB dataset. The final MI classification accuracy reached 82.01% in IIA dataset and 87.52% in IIB, which outperformed the state-of-the-art methods in the literature.
The Electroencephalogram (EEG)-based emotion recognition is promising yet limited by the requirement of a large number of training data. Collecting substantial labeled samples in the training trails is the key to the generalization on the test trails. This process is time-consuming and laborious. In recent years, several studies have proposed various semi-supervised learning (e.g., active learning) and transfer learning (e.g., domain adaptation, style transfer mapping) methods to alleviate the requirement on training data. However, most of them are iterative methods, which need considerable training time and are unfeasible in practice. To tackle this problem, we present the Fast Online Instance Transfer (FOIT) for improved affective Brain-computer Interface (aBCI). FOIT selects auxiliary data from historical sessions and (or) other subjects heuristically, which are then combined with the training data for supervised training. The predictions on the test trails are made by the multiclassifier ensemble. As a one-shot algorithm, FOIT avoids time-consuming iterations. Experimental results show that FOIT brings significant improvement in accuracy for the three category classification (1%-8%) on the SEED dataset and four-category classification (1%-14%) on the SEED-IV dataset in the cross-subject, cross-session, and cross-all scenarios. The time cost over the baselines is moderate (~35s in average for our machine). To achieve the comparative accuracies, the iterative methods require much more time (~45s-~900s). FOIT provides a simple, fast, and practically feasible solution to improve the generalization of aBCIs and allows various choices of classifiers without constrains. Our codes are available online.
Abstract: The paper presented the easiest way to the neuroscience world with a shield for RaspberryPI - PIEEG. Over the past decade, advances in artificial intelligence have expanded and reshaped many industries. It also affected neuroscience, where machine learning made possible to find practical use for electroencephalogram signals in the brain in various fields of engineering. As a result, popularity of brain-computer interface devices used for recording brain signals at low-cost has increased dramatically. In recent years, several companies started manufacturing devices capable of recording EEG signals using non-invasive electrodes at low cost. Even then, the cost of these devices often exceeds IOOOUSD. In this paper, we aim for yet further cost reduction by proposing a Raspberry PI -based device for reading EEG signal. According to its characteristics, the device is compared to the signal-to-noise level of the ADS1299 device produced by Texas instruments and used for comparison in this paper. The proposed device, on top of satisfying noise level and accuracy of detected artifacts in the signal, was also tested and confirmed to successfully resist the effect of electromagnetic interference, successfully detect the alpha brain wave, and detect the P300 paradigm.
Neural population activity relating to behaviour is assumed to be inherently low-dimensional despite the observed high dimensionality of data recorded using multi-electrode arrays. Therefore, predicting behaviour from neural population recordings has been shown to be most effective when using latent variable models. Over time however, the activity of single neurons can drift, and different neurons will be recorded due to movement of implanted neural probes. This means that a decoder trained to predict behaviour on one day performs worse when tested on a different day. On the other hand, evidence suggests that the latent dynamics underlying behaviour may be stable even over months and years. Based on this idea, we introduce a model capable of inferring behaviourally relevant latent dynamics from previously unseen data recorded from the same animal, without any need for decoder recalibration. We show that unsupervised domain adaptation combined with a sequential variational autoencoder, trained on several sessions, can achieve good generalisation to unseen data and correctly predict behaviour where conventional methods fail. Our results further support the hypothesis that behaviour-related neural dynamics are low-dimensional and stable over time, and will enable more effective and flexible use of brain computer interface technologies.
The method of Common Spatial Patterns (CSP) is widely used for feature extraction of electroencephalography (EEG) data, such as in motor imagery brain-computer interface (BCI) systems. It is a data-driven method estimating a set of spatial filters so that the power of the filtered EEG signal is maximized for one motor imagery class and minimized for the other. This method, however, is prone to overfitting and is known to suffer from poor generalization especially with limited calibration data. Additionally, due to the high heterogeneity in brain data and the non-stationarity of brain activity, CSP is usually trained for each user separately resulting in long calibration sessions or frequent re-calibrations that are tiring for the user. In this work, we propose a novel algorithm called Spectrally Adaptive Common Spatial Patterns (SACSP) that improves CSP by learning a temporal/spectral filter for each spatial filter so that the spatial filters are concentrated on the most relevant temporal frequencies for each user. We show the efficacy of SACSP in providing better generalizability and higher classification accuracy from calibration to online control compared to existing methods. Furthermore, we show that SACSP provides neurophysiologically relevant information about the temporal frequencies of the filtered signals. Our results highlight the differences in the motor imagery signal among BCI users as well as spectral differences in the signals generated for each class, and show the importance of learning robust user-specific features in a data-driven manner.
While Artificial Intelligence (AI) technologies are being progressively developed, artists and researchers are investigating their role in artistic practices. In this work, we present an AI-based Brain-Computer Interface (BCI) in which humans and machines interact to express feelings artistically. This system and its production of images give opportunities to reflect on the complexities and range of human emotions and their expressions. In this discussion, we seek to understand the dynamics of this interaction to reach better co-existence in fairness, inclusion, and aesthetics.
This paper presents Open-source software and a developed shield board for the Raspberry Pi family of single-board computers that can be used to read EEG signals. We have described the mechanism for reading EEG signals and decomposing them into a Fourier series and provided examples of controlling LEDs and a toy robot by blinking. Finally, we discussed the prospects of the brain-computer interface for the near future and considered various methods for controlling external mechanical objects using real-time EEG signals.
Significance: Optical neuroimaging has become a well-established clinical and research tool to monitor cortical activations in the human brain. It is notable that outcomes of functional Near-InfraRed Spectroscopy (fNIRS) studies depend heavily on the data processing pipeline and classification model employed. Recently, Deep Learning (DL) methodologies have demonstrated fast and accurate performances in data processing and classification tasks across many biomedical fields. Aim: We aim to review the emerging DL applications in fNIRS studies. Approach: We first introduce some of the commonly used DL techniques. Then the review summarizes current DL work in some of the most active areas of this field, including brain-computer interface, neuro-impairment diagnosis, and neuroscience discovery. Results: Of the 63 papers considered in this review, 32 report a comparative study of deep learning techniques to traditional machine learning techniques where 26 have been shown outperforming the latter in terms of classification accuracy. Additionally, 8 studies also utilize deep learning to reduce the amount of preprocessing typically done with fNIRS data or increase the amount of data via data augmentation. Conclusions: The application of DL techniques to fNIRS studies has shown to mitigate many of the hurdles present in fNIRS studies such as lengthy data preprocessing or small sample sizes while achieving comparable or improved classification accuracy.
Dealing with irregular domains, graph signal processing (GSP) has attracted much attention especially in brain imaging analysis. Motor imagery tasks are extensively utilized in brain-computer interface (BCI) systems that perform classification using features extracted from Electroencephalogram signals. In this paper, a GSP-based approach is presented for two-class motor imagery tasks classification. The proposed method exploits simultaneous diagonalization of two matrices that quantify the covariance structure of graph spectral representation of data from each class, providing a discriminative subspace where distinctive features are extracted from the data. The performance of the proposed method was evaluated on Dataset IVa from BCI Competition III. Experimental results show that the proposed method outperforms two state-of-the-art alternative methods.