Recently, visual perception (VP) and visual imagery (VI) paradigms are investigated in several brain-computer interface (BCI) studies. VP and VI are defined as a changing of brain signals when perceiving and memorizing visual information, respectively. These paradigms could be alternatives to the previous visual-based paradigms which have limitations such as fatigue and low information transfer rates (ITR). In this study, we analyzed VP and VI to investigate the possibility to control BCI. First, we conducted a time-frequency analysis with event-related spectral perturbation. In addition, two types of decoding accuracies were obtained with convolutional neural network to verify whether the brain signals can be distinguished from each class in the VP and whether they can be differentiated with VP and VI paradigms. As a result, the 6-class classification performance in VP was 32.56% and the binary classification performance which classifies two paradigms was 90.16%.
A scalp-recording electroencephalography (EEG)-based brain-computer interface (BCI) system can greatly improve the quality of life for people who suffer from motor disabilities. Deep neural networks consisting of multiple convolutional, LSTM and fully-connected layers are created to decode EEG signals to maximize the human intention recognition accuracy. However, prior FPGA, ASIC, ReRAM and photonic accelerators cannot maintain sufficient battery lifetime when processing real-time intention recognition. In this paper, we propose an ultra-low-power photonic accelerator, MindReading, for human intention recognition by only low bit-width addition and shift operations. Compared to prior neural network accelerators, to maintain the real-time processing throughput, MindReading reduces the power consumption by 62.7\% and improves the throughput per Watt by 168\%.
Brain-computer interface (BCI) systems have potential as assistive technologies for individuals with severe motor impairments. Nevertheless, individuals must first participate in many training sessions to obtain adequate data for optimizing the classification algorithm and subsequently acquiring brain-based control. Such traditional training paradigms have been dubbed unengaging and unmotivating for users. In recent years, it has been shown that the synergy of virtual reality (VR) and a BCI can lead to increased user engagement. This study created a 3-class BCI with a rather elaborate EEG signal processing pipeline that heavily utilizes machine learning. The BCI initially presented sham feedback but was eventually driven by EEG associated with motor imagery. The BCI tasks consisted of motor imagery of the feet and left and right hands, which were used to navigate a single-path maze in VR. Ten of the eleven recruited participants achieved online performance superior to chance (p < 0.01), while the majority successfully completed more than 70% of the prescribed navigational tasks. These results indicate that the proposed paradigm warrants further consideration as neurofeedback BCI training tool. A paradigm that allows users, from their perspective, control from the outset without the need for prior data collection sessions.
Brain computer interface (BCI) has been popular as a key approach to monitor our brains recent year. Mental states monitoring is one of the most important BCI applications and becomes increasingly accessible. However, the mental state prediction accuracy and generality through encephalogram (EEG) are not good enough for everyday use. Here in this paper we present a deep learning-based EEG decoding method to read mental states. We propose a novel 1D convolutional neural network with different filter lengths to capture different frequency bands information. To improve the prediction accuracy, we also used a resnet-like structure to train a relatively deep convolutional neural network to promote feature extraction. Compared with traditional ways of predictions such as KNN and SVM, we achieved a significantly better result with an accuracy of 96.40%. Also, in contrast with some already published open source deep neural network structures, our methods achieved the state of art prediction accuracy on a mental state recognition dataset. Our results demonstrate using only 1D convolution could extract the features of EEG and the possibility of mental state prediction using portable EEG devices.
In this paper, we evaluate a semi-autonomous brain-computer interface (BCI) for manipulation tasks. In such system, the user controls a robotic arm through motor imagery commands. In traditional process-control BCI systems, the user has to provide those commands continuously in order manipulate the effector of the robot step-by-step, which results in a tiresome process for simple tasks such as pick and replace an item from a surface. Here, we take a semi-autonomous approach based on a conformal geometric algebra model that solves the inverse kinematics of the robot on the fly, then the user only has to decide on the start of the movement and the final position of the effector (goal-selection approach). Under these conditions, we implemented pick-and-place tasks with a disk as an item and two target areas placed on the table at arbitrary positions. An artificial vision (AV) algorithm was used to obtain the positions of the items expressed in the robot frame through images captured with a webcam. Then, the AV algorithm is integrated to the inverse kinematics model to perform the manipulation tasks. As proof-of-concept, different users were trained to control the pick-and-place tasks through the process-control and semi-autonomous goal-selection approaches, so that the performance of both schemes could be compared. Our results show the superiority in performance of the semi-autonomous approach, as well as evidence of less mental fatigue with it.
We proposed a novel visual stimulus for brain-computer interface. The stimulus is in the form gaiting sequence of a human. The hypothesis is that observing such a visual stimulus would simultaneously induce 1) steady-state motion visual evoked potential (SSMVEP) in the occipital area, similarly to an SSVEP stimulus; and 2) sensorimotor rhythm (SMR) in the primary sensorimotor area, because such action observation (AO) could activate the mirror neuron system. Canonical correlation analysis (CCA) was used to detect SSMVEP from occipital EEG, and event-related spectral perturbations (ERSP) were used to identify SMR in the EEG from the sensorimotor area. The results showed that the proposed visual gaiting stimulus-induced SSMVEP, with classification accuracies of 88.9 $\pm$ 12.0% in a four-class scenario. More importantly, it induced clear and sustained event-related desynchronization/synchronization (ERD/ERS) in the EEG from the sensorimotor area, while no ERD/ERS in the sensorimotor area could be observed when the other two SSVEP stimuli were used. Further, for participants with sufficiently clear SSMVEP pattern (classification accuracy > 85%), the ERD index values in mu-beta band induced by the proposed gaiting stimulus were statistically different from that of the other two types of stimulus. Therefore, a novel BCI based on the proposed stimulus has potential in neurorehabilitation applications because it simultaneously has the high accuracy of an SSMVEP (~90% accuracy in a four-class setup) and the ability to activate sensorimotor cortex. And such potential will be further explored in future studies.
Brain computer interface based assistive technology are currently promoted for motor rehabilitation of the neuromuscular ailed individuals. Recent studies indicate a high potential of utilising electroencephalography (EEG) to extract motor related intentions. Limbic movement intentions are already exhaustively studied by the researchers with high accuracy rate. But, capturing movement of fingers from EEG is still in nascent stage. In this study, we have proposed an ensemble learning based approach for EEG in distinguishing between movements of different fingers, namely, thumb, index, and middle. Six healthy subjects participated in this study. Common spatial patterns (CSP) were extracted as features to classify with the extra tree or extremely randomized tree binary classifier. The average classification accuracy of decoding a finger from rest condition was found to be $74\%$, wheres in discriminating of movement of pair of fingers average accuracy was $60\%$. Furthermore, error correcting output coding (ECOC) was added to the binary classifier to use it in multiclass classification. The proposed algorithm achieved a maximum kappa value of 0.36 among the subjects.
The oddball paradigm is widely applied to the investigation of multiple cognitive functions. Prior studies have explored the cortical oscillation and power spectral differing from the resting-state conduction to oddball paradigm, but whether brain networks existing the significant difference is still unclear. Our study addressed how the brain reconfigures its architecture from a resting-state condition (i.e., baseline) to P300 stimulus task in the visual oddball paradigm. In this study, electroencephalogram (EEG) datasets were collected from 24 postgraduate students, who were required to only mentally count the number of target stimulus; afterwards the functional EEG networks constructed in different frequency bands were compared between baseline and oddball task conditions to evaluate the reconfiguration of functional network in the brain. Compared to the baseline, our results showed the significantly (p < 0.05) enhanced delta/theta EEG connectivity and decreased alpha default mode network in the progress of brain reconfiguration to the P300 task. Furthermore, the reconfigured coupling strengths were demonstrated to relate to P300 amplitudes, which were then regarded as input features to train a classifier to differentiate the high and low P300 amplitudes groups with an accuracy of 77.78%. The findings of our study help us to understand the changes of functional brain connectivity from resting-state to oddball stimulus task, and the reconfigured network pattern has the potential for the selection of good subjects for P300-based brain- computer interface.
Humans perform remarkably well in many cognitive tasks including pattern recognition. However, the neuronal mechanisms underlying this process are not well understood. Nevertheless, artificial neural networks, inspired in brain circuits, have been designed and used to tackle spatio-temporal pattern recognition tasks. In this paper we present a multineuronal spike pattern detection structure able to autonomously implement online learning and recognition of parallel spike sequences (i.e., sequences of pulses belonging to different neurons/neural ensembles). The operating principle of this structure is based on two spiking/synaptic neurocomputational characteristics: spike latency, that enables neurons to fire spikes with a certain delay and heterosynaptic plasticity, that allows the own regulation of synaptic weights. From the perspective of the information representation, the structure allows mapping a spatio-temporal stimulus into a multidimensional, temporal, feature space. In this space, the parameter coordinate and the time at which a neuron fires represent one specific feature. In this sense, each feature can be considered to span a single temporal axis. We applied our proposed scheme to experimental data obtained from a motor inhibitory cognitive task. The test exhibits good classification performance, indicating the adequateness of our approach. In addition to its effectiveness, its simplicity and low computational cost suggest a large scale implementation for real time recognition applications in several areas, such as brain computer interface, personal biometrics authentication or early detection of diseases.
A user of Brain Computer Interface (BCI) system must be able to control external computer devices with brain activity. Although the proof-of-concept was given decades ago, the reliable translation of user intent into device control commands is still a major challenge. There are problems associated with classification of different BCI tasks. In this paper we propose the use of chaotic indices of the BCI. We use largest Lyapunov exponent, mutual information, correlation dimension and minimum embedding dimension as the features for the classification of EEG signals which have been released by BCI Competition IV. A multi-layer Perceptron classifier and a KM- SVM(support vector machine classifier based on k-means clustering) is used for classification process, which lead us to an accuracy of 95.5%, for discrimination between two motor imagery tasks.
Noninvasive EEG (electroencephalography) based auditory attention detection could be useful for improved hearing aids in the future. This work is a novel attempt to investigate the feasibility of online modulation of sound sources by probabilistic detection of auditory attention, using a noninvasive EEG-based brain computer interface. Proposed online system modulates the upcoming sound sources through gain adaptation which employs probabilistic decisions (soft decisions) from a classifier trained on offline calibration data. In this work, calibration EEG data were collected in sessions where the participants listened to two sound sources (one attended and one unattended). Cross-correlation coefficients between the EEG measurements and the attended and unattended sound source envelope (estimates) are used to show differences in sharpness and delays of neural responses for attended versus unattended sound source. Salient features to distinguish attended sources from the unattended ones in the correlation patterns have been identified, and later they have been used to train an auditory attention classifier. Compared to the existing results in the literature, in this paper we have two main contributions. First, using the auditory attention classifier, we have shown high offline detection performance with single channel EEG measurements of shorter duration compared to the existing approaches in the literature which employ large number of channels with longer EEG measurements. Second, using the classifier trained offline in the calibration session, we have shown the performance of the online sound source modulation system. We observe that online sound source modulation system is able to keep the level of attended sound source higher than the unattended source. Keywords: auditory BCI, cocktail party problem, auditory attention classification
The paper presents a study of two novel visual motion onset stimulus-based brain-computer interfaces (vmoBCI). Two settings are compared with afferent and efferent to a computer screen center motion patterns. Online vmoBCI experiments are conducted in an oddball event-related potential (ERP) paradigm allowing for "aha-responses" decoding in EEG brainwaves. A subsequent stepwise linear discriminant analysis classification (swLDA) classification accuracy comparison is discussed based on two inter-stimulus-interval (ISI) settings of 700 and 150 ms in two online vmoBCI applications with six and eight command settings. A research hypothesis of classification accuracy non-significant differences with various ISIs is confirmed based on the two settings of 700 ms and 150 ms, as well as with various numbers of ERP response averaging scenarios. The efferent in respect to display center visual motion patterns allowed for a faster interfacing and thus they are recommended as more suitable for the no-eye-movements requiring visual BCIs.
To study body ownership and control, illusions that elicit these feelings in non-body objects are widely used. Classically introduced with the Rubber Hand Illusion, these illusions have been replicated more recently in virtual reality and by using brain-computer interfaces. Traditionally these illusions investigate the replacement of a body part by an artificial counterpart, however as brain-computer interface research develops it offers us the possibility to explore the case where non-body objects are controlled in addition to movements of our own limbs. Therefore we propose a new illusion designed to test the feeling of ownership and control of an independent supernumerary hand. Subjects are under the impression they control a virtual reality hand via a brain-computer interface, but in reality there is no causal connection between brain activity and virtual hand movement but correct movements are observed with 80% probability. These imitation brain-computer interface trials are interspersed with movements in both the subjects' real hands, which are in view throughout the experiment. We show that subjects develop strong feelings of ownership and control over the third hand, despite only receiving visual feedback with no causal link to the actual brain signals. Our illusion is crucially different from previously reported studies as we demonstrate independent ownership and control of the third hand without loss of ownership in the real hands.
We present results of an approach to a code-modulated visual evoked potential (cVEP) based brain-computer interface (BCI) paradigm using four high-frequency flashing stimuli. To generate higher frequency stimulation compared to the state-of-the-art cVEP-based BCIs, we propose to use the light-emitting diodes (LEDs) driven from a small micro-controller board hardware generator designed by our team. The high-frequency and green-blue chromatic flashing stimuli are used in the study in order to minimize a danger of a photosensitive epilepsy (PSE). We compare the the green-blue chromatic cVEP-based BCI accuracies with the conventional white-black flicker based interface.
The paper presents a study of an inter-stimulus interval (ISI) influence on a tactile point-pressure stimulus-based brain-computer interface's (tpBCI) classification accuracy. A novel tactile pressure generating tpBCI stimulator is also discussed, which is based on a three-by-three pins' matrix prototype. The six pin-linear patterns are presented to the user's palm during the online tpBCI experiments in an oddball style paradigm allowing for "the aha-responses" elucidation, within the event related potential (ERP). A subsequent classification accuracies' comparison is discussed based on two ISI settings in an online tpBCI application. A research hypothesis of classification accuracies' non-significant differences with various ISIs is confirmed based on the two settings of 120 ms and 300 ms, as well as with various numbers of ERP response averaging scenarios.
This study provides a comprehensive test of a head-related impulse response (HRIR) cues for a spatial auditory brain-computer interface (saBCI) speller paradigm. We present a comparison with the conventional virtual sound headphone-based spatial auditory modality. We propose and optimize the three types of sound spatialization settings using a variable elevation in order to evaluate the HRIR efficacy for the saBCI. Three experienced and seven naive BCI users participated in the three experimental setups based on ten presented Japanese syllables. The obtained EEG auditory evoked potentials (AEP) resulted with encouragingly good and stable P300 responses in online BCI experiments. Our case study indicated that users could perceive elevation in the saBCI experiments generated using the HRIR measured from a general head model. The saBCI accuracy and information transfer rate (ITR) scores have been improved comparing to the classical horizontal plane-based virtual spatial sound reproduction modality, as far as the healthy users in the current pilot study are concerned.
This paper presents an applied concept of a brain-computer interface (BCI) student research laboratory (BCI-LAB) at the Life Science Center of TARA, University of Tsukuba, Japan. Several successful case studies of the student projects are reviewed together with the BCI Research Award 2014 winner case. The BCI-LAB design and project-based teaching philosophy is also explained. Future teaching and research directions summarize the review.
We present an auditory stimulus optimization and a pilot study of a two-step input speller application combined with a spatial auditory brain-computer interface (saBCI) for paralyzed users. The application has been developed for 45, out of 48 defining the full set, Japanese kana characters in a two-step input procedure setting for an easy-to-use BCI-speller interface. The user first selects the representative letter of a subset, defining the second step. In the second step, the final choice is made. At each interfacing step, the choices are classified based on the P300 event related potential (ERP) responses captured in the EEG, as in the classic oddball paradigm. The BCI online experiment and EEG responses classification results of the pilot study confirm the effectiveness of the proposed spelling method.
This study reports on a head related impulse response (HRIR) application to an auditory spatial brain-computer interface (BCI) speller paradigm. Six experienced and five BCI-naive users participated in an experimental spelling set up based on five Japanese vowels. Obtained auditory evoked potentials resulted with encouragingly good and stable P300-responses in online BCI experiments. Our case study indicated that the headphone reproduced auditory (HRIR- based) spatial sound paradigm could be a viable alternative to the established multi-loudspeaker surround sound BCI-speller applications, as far as healthy pilot study users are concerned.
This paper provides a proof of concept for an EEG-based reconstruction of a visual image which is on a user's mind. Our approach is based on the Rapid Serial Visual Presentation (RSVP) of polygon primitives and Brain-Computer Interface (BCI) technology. The presentation of polygons that contribute to build a target image (because they match the shape and/or color of the target) trigger attention-related EEG patterns. Accordingly, these target primitives can be determined using BCI classification of Event-Related Potentials (ERPs). They are then accumulated in the display until a satisfactory reconstruction is reached. Selection steps have an average classification accuracy of $75\%$. $25\%$ of the images could be reconstructed completely, while more than $65\%$ of the available visual details could be captured on average. Most of the misclassifications were not misinterpretations of the BCI concerning users' intent; rather, users tried to select polygons that were different than what was intended by the experimenters. Open problems and alternatives to develop a practical BCI-based image reconstruction application are discussed.