In the past decade, deep learning algorithms gained a remarkable interest in the signal processing community. The availability of big datasets and advanced computational resources resulted in developing efficient algorithms. However, such algorithms are biased towards the training dataset. Thus, the transferability of deep-learning-based operators are challenging, especially when the goal is to apply the learned operator on a new dataset/domain. Lack of transferability of learned operator across domains hinders the applicability of deep learning algorithms in processing seismic data. Unlike camera images, the comprehensively labeled seismic datasets are not available. Moreover, from one task to another, the training parameters should be tuned. To remedy this shortcoming, we have developed a workflow that transfers the learned operator from the camera images to the seismic domain, without modifying its training parameters. The similarities in the algorithms and optimization methods in camera and seismic data processing allow us to do so. Accordingly, by incorporating feed-forward de-noising convolutional neural networks (DnCNN) in regularization by de-noising regularizer, we formulate two transferable optimization problems for de-noising and compressive sensing recovery of seismic data. Simulated and real-world data examples show the efficiency of our proposed workflow.
Wireless communications are envisioned to bring about dramatic changes in the future, with a variety of emerging applications, such as virtual reality (VR), Internet of things (IoT), etc., becoming a reality. However, these compelling applications have imposed many new challenges, including unknown channel models, low-latency requirement in large-scale super-dense networks, etc. The amazing success of deep learning (DL) in various fields, particularly in computer science, has recently stimulated increasing interest in applying it to address those challenges. Hence, in this review, a pair of dominant methodologies of using DL for wireless communications are investigated. The first one is DL-based architecture design, which breaks the classical model-based block design rule of wireless communications in the past decades. The second one is DL-based algorithm design, which will be illustrated by several examples in a series of typical techniques conceived for 5G and beyond. Their principles, key features, and performance gains will be discussed. Furthermore, open problems and future research opportunities will also be pointed out, highlighting the interplay between DL and wireless communications. We expect that this review can stimulate more novel ideas and exciting contributions for intelligent wireless communications.
Deep learning has made great strides lately with the availability of powerful computing machines and the advent of user-friendly programming environments. It is anticipated that the deep learning algorithms will entirely provision the majority of operations in 6G. One such environment where deep learning can be the right solution is load balancing in future 6G intelligent wireless networks. Load balancing presents an efficient, cost-effective method to improve the data process capability, throughput, and expand the bandwidth, thus enhancing the adaptability and availability of networks. Hence a load balancing algorithm based on Long Short Term Memory(LSTM) deep neural network is proposed through which the coverage area of base station changes according to geographic traffic distribution, catering the requirement for future generation 6G heterogeneous network. The LSTM model performance is evaluated by considering three different scenarios, and the results were presented. Load variance coefficient(LVC) and load factor(LF) are introduced and validated over two wireless network layouts(WNL) to study the Quality of Service(QoS) and load distribution. The proposed method shows a decrease of LVC by 98.311% and 99.21% for WNL1, WNL2 respectively.
We investigate solving partial integro-differential equations (PIDEs) using unsupervised deep learning in this paper. To price options, assuming underlying processes follow Levy processes, we require to solve PIDEs. In supervised deep learning, pre-calculated labels are used to train neural networks to fit the solution of the PIDE. In an unsupervised deep learning, neural networks are employed as the solution, and the derivatives and the integrals in the PIDE are calculated based on the neural network. By matching the PIDE and its boundary conditions, the neural network gives an accurate solution of the PIDE. Once trained, it would be fast for calculating options values as well as option Greeks.
Decoding brain signals has gained many attention and has found much applications in recent years such as Brain Computer Interfaces, communicating with controlling external devices using the user's intentions, occupies an emerging field with the potential of changing the world, with diverse applications from rehabilitation to human augmentation. This being said brain signal analysis, EEG brain signal analysis in particular, is a challenging task. With the advances and achievements in the field of deep learning in problem solving with using only raw data, few attempts has been carried in recent years, to apply deep learning to tackle EEG among other types of brain signals. In this study, we propose a novel loss function, called DeepCSP to extend the classical Common Spatial Patterns to a non linear, differentiable module to serve as the loss function to enforce linearly separable latent representations of EEG signals belonging to different classes in an end to end manner on raw signals without the need to perform extensive feature engineering. With recent generalizations of deep learning methods to work on arbitrarily structured graphs and the introduced loss we have proposed two light weight models to decode EEG signals and carried experiments to show their performance.
Pedestrian tracking has long been considered an important problem, especially in security applications. Previously,many approaches have been proposed with various types of sensors. One popular method is Pedestrian Dead Reckoning(PDR) [1] which is based on the inertial measurement unit(IMU) sensor. However PDR is an integration and threshold based method, which suffers from accumulation errors and low accuracy. In this paper, we propose a novel method in which the sensor data is fed into a deep learning model to predict the displacements and orientations of the pedestrian. We also devise a new apparatus to collect and construct databases containing synchronized IMU sensor data and precise locations measured by a LIDAR. The preliminary results are promising, and we plan to push this forward by collecting more data and adapting the deep learning model for all general pedestrian motions.
Building the future profit and loss (P&L) distribution of a portfolio holding, among other assets, highly non-linear and path-dependent derivatives is a challenging task. We provide a simple machinery where more and more assets could be accounted for in a simple and semi-automatic fashion. We resort to a variation of the Least Square Monte Carlo algorithm where interpolation of the continuation value of the portfolio is done with a feed forward neural network. This approach has several appealing features not all of them will be fully discussed in the paper. Neural networks are extremely flexible regressors. We do not need to worry about the fact that for multi assets payoff, the exercise surface could be non connected. Neither we have to search for smart regressors. The idea is to use, regardless of the complexity of the payoff, only the underlying processes. Neural networks with many outputs can interpolate every single assets in the portfolio generated by a single Monte Carlo simulation. This is an essential feature to account for the P&L distribution of the whole portfolio when the dependence structure between the different assets is very strong like the case where one has contingent claims written on the same underlying.
Deep learning methods have permeated into the research area of computer-aided drug design. The deep learning generative model and classical algorithm can be simultaneously used for three-dimensional (3D) drug design in the 3D pocket of the receptor. Here, three aspects of MolAICal are illustrated for drug design: in the first part, the MolAICal uses the genetic algorithm, Vinardo score and deep learning generative model trained by generative adversarial net (GAN) for drug design. In the second part, the deep learning generative model is trained by drug-like molecules from the drug database such as ZINC database. The MolAICal invokes the deep learning generative model and molecular docking for drug virtual screening automatically. In the third part, the useful drug tools are added for calculating the relative properties such as Pan-assay interference compounds (PAINS), Lipinski's rule of five, synthetic accessibility (SA), and so on. Besides, the structural similarity search and quantitative structure-activity relationship (QSAR), etc are also embedded for the calculations of drug properties in the MolAICal. MolAICal will constantly optimize and develop the current and new modules for drug design. The MolAICal can help the scientists, pharmacists and biologists to design the rational 3D drugs in the receptor pocket through the deep learning model and classical programming. MolAICal is free of charge for any academic and educational purposes, and it can be downloaded from the website https://molaical.github.io.
Deep learning has been recently applied to many problems in wireless communications including modulation classification and symbol decoding. Many of the existing end-to-end learning approaches demonstrated robustness to signal distortions like frequency and timing errors, and outperformed classical signal processing techniques with sufficient training. However, deep learning approaches typically require hundreds of thousands of floating points operations for inference, which is orders of magnitude higher than classical signal processing approaches and thus do not scale well for long sequences. Additionally, they typically operate as a black box and without insight on how their final output was obtained, they can't be integrated with existing approaches. In this paper, we propose a novel neural network architecture that combines deep learning with linear signal processing typically done at the receiver to realize joint modulation classification and symbol recovery. The proposed method estimates signal parameters by learning and corrects signal distortions like carrier frequency offset and multipath fading by linear processing. Using this hybrid approach, we leverage the power of deep learning while retaining the efficiency of conventional receiver processing techniques for long sequences. The proposed hybrid approach provides good accuracy in signal distortion estimation leading to promising results in terms of symbol error rate. For modulation classification accuracy, it outperforms many state of the art deep learning networks.
Since the signal with strong power should be demodulated first for successive interference cancellation (SIC) demodulation in non-orthogonal multiple access (NOMA) systems, the base station (BS) should inform the near user terminal (UT), which has allocated higher power, of modulation mode of the far user terminal. To avoid unnecessary signaling overhead in this process, a blind detection algorithm of NOMA signal modulation mode is designed in this paper. Taking the joint constellation density diagrams of NOMA signal as the detection features, deep residual network is built for classification, so as to detect the modulation mode of NOMA signal. In view of the fact that the joint constellation diagrams are easily polluted by high intensity noise and lose their real distribution pattern, the wavelet denoising method is adopted to improve the quality of constellations. The simulation results represent that the proposed algorithm can achieve satisfactory detection accuracy in NOMA systems. In addition, the factors affecting the recognition performance are also verified and analyzed.
Mechanical properties of tissue provide valuable information for identifying lesions. One approach to obtain quantitative estimates of elastic properties is shear wave elastography with optical coherence elastography (OCE). However, given the shear wave velocity, it is still difficult to estimate elastic properties. Hence, we propose deep learning to directly predict elastic tissue properties from OCE data. We acquire 2D images with a frame rate of 30 kHz and use convolutional neural networks to predict gelatin concentration, which we use as a surrogate for tissue elasticity. We compare our deep learning approach to predictions from conventional regression models, using the shear wave velocity as a feature. Mean absolut prediction errors for the conventional approaches range from 1.32$\pm$0.98 p.p. to 1.57$\pm$1.30 p.p. whereas we report an error of 0.90$\pm$0.84 p.p for the convolutional neural network with 3D spatio-temporal input. Our results indicate that deep learning on spatio-temporal data outperforms elastography based on explicit shear wave velocity estimation.
In this paper, an efficient massive multiple-input multiple-output (MIMO) detector is proposed by employing a deep neural network (DNN). Specifically, we first unfold an existing iterative detection algorithm into the DNN structure, such that the detection task can be implemented by deep learning (DL) approach. We then introduce two auxiliary parameters at each layer to better cancel multiuser interference (MUI). The first parameter is to generate the residual error vector while the second one is to adjust the relationship among previous layers. We further design the training procedure to optimize the auxiliary parameters with pre-processed inputs. The so derived MIMO detector falls into the category of model-driven DL. The simulation results show that the proposed MIMO detector can achieve preferable detection performance compared to the existing detectors for massive MIMO systems.
Positioning is becoming a key component in many Internet of Things (IoT) applications. The main challenges and limitations are the narrow bandwidth, low power and low cost which reduces the accuracy of the time of arrival (TOA) estimation. In this paper, we consider the positioning scenario of Narrowband IoT (NB-IoT) that can benefit from observed time difference of arrival (OTDOA). By applying the deep learning based technique, we explore the generalization and feature extraction abilities of neural networks to tackle the aforementioned challenges. As demonstrated in the numerical experiments, the proposed algorithm can be used in different inter-site distance situations and results in a 15% and 50% positioning accuracy improvement compared with Gauss-Newton method in line-of-sight (LOS) scenario and non-line-of-sight (NLOS) scenario respectively.
This paper presents a review of multi-objective deep learning methods that have been introduced in the literature for speech denoising. After stating an overview of conventional, single objective deep learning, and hybrid or combined conventional and deep learning methods, a review of the mathematical framework of the multi-objective deep learning methods for speech denoising is provided. A representative method from each speech denoising category, whose codes are publicly available, is selected and a comparison is carried out by considering the same public domain dataset and four widely used objective metrics. The comparison results indicate the effectiveness of the multi-objective method compared with the other methods, in particular when the signal-to-noise ratio is low. Possible future improvements that can be achieved are also mentioned.
With recent rapid advances in photonic integrated circuits, it has been demonstrated that programmable photonic chips can be used to implement artificial neural networks. Convolutional neural networks (CNN) are a class of deep learning methods that have been highly successful in applications such as image classification and speech processing. We present an architecture to implement a photonic CNN using the Fourier transform property of integrated star couplers. We show, in computer simulation, high accuracy image classification using the MNIST dataset. We also model component imperfections in photonic CNN and show that the performance degradation can be recovered in a programmable chip. Our proposed architecture provides a large reduction in physical footprint compared to current implementations as it utilizes the natural advantages of optics and hence offers a scalable pathway towards integrated photonic deep learning processors.
Partly due to the use of exhaustive-annotated data, deep networks have achieved impressive performance on medical image segmentation. Medical imaging data paired with noisy annotation are, however, ubiquitous, but little is known about the effect of noisy annotation on deep learning-based medical image segmentation. We studied the effects of noisy annotation in the context of mandible segmentation from CT images. First, 202 images of Head and Neck cancer patients were collected from our clinical database, where the organs-at-risk were annotated by one of 12 planning dosimetrists. The mandibles were roughly annotated as the planning avoiding structure. Then, mandible labels were checked and corrected by a physician to get clean annotations. At last, by varying the ratios of noisy labels in the training data, deep learning-based segmentation models were trained, one for each ratio. In general, a deep network trained with noisy labels had worse segmentation results than that trained with clean labels, and fewer noisy labels led to better segmentation. When using 20% or less noisy cases for training, no significant difference was found on the prediction performance between the models trained by noisy or clean. This study suggests that deep learning-based medical image segmentation is robust to noisy annotations to some extent. It also highlights the importance of labeling quality in deep learning
Poor air quality has become an increasingly critical challenge for many metropolitan cities, which carries many catastrophicphysical and mental consequences on human health and quality of life. However, accurately monitoring and forecasting air qualityremains a highly challenging endeavour. Limited by geographically sparse data, traditional statistical models and newly emergingdata-driven methods of air quality forecasting mainly focused on the temporal correlation between the historical temporal datasets of airpollutants. However, in reality, both distribution and dispersion of air pollutants are highly location-dependant. In this paper, we proposea novel hybrid deep learning model that combines Convolutional Neural Networks (CNN) and Long Short Term Memory (LSTM)together to forecast air quality at high-resolution. Our model can utilize the spatial correlation characteristic of our air pollutant datasetsto achieve higher forecasting accuracy than existing deep learning models of air pollution forecast.
The automatic assessment of hippocampus volume is an important tool in the study of several neurodegenerative diseases such as Alzheimer's disease. Specifically, the measurement of hippocampus subfields properties is of great interest since it can show earlier pathological changes in the brain. However, segmentation of these subfields is very difficult due to their complex structure and for the need of high-resolution magnetic resonance images manually labeled. In this work, we present a novel pipeline for automatic hippocampus subfield segmentation based on a deeply supervised convolutional neural network. Results of the proposed method are shown for two available hippocampus subfield delineation protocols. The method has been compared to other state-of-the-art methods showing improved results in terms of accuracy and execution time.
In this paper we introduce a deep learning method for pricing and hedging American-style options. It first computes a candidate optimal stopping policy. From there it derives a lower bound for the price. Then it calculates an upper bound, a point estimate and confidence intervals. Finally, it constructs an approximate dynamic hedging strategy. We test the approach on different specifications of a Bermudan max-call option. In all cases it produces highly accurate prices and dynamic hedging strategies with small replication errors.
Quantitative susceptibility mapping (QSM) has gained broad interests in the field by extracting biological tissue properties, predominantly myelin, iron and calcium from magnetic resonance imaging (MRI) phase measurements in vivo. Thereby, QSM can reveal pathological changes of these key components in a variety of diseases. QSM requires multiple processing steps such as phase unwrapping, background field removal and field-to-source-inversion. Current state of the art techniques utilize iterative optimization procedures to solve the inversion and background field correction, which are computationally expensive and require a careful choice of regularization parameters. With the recent success of deep learning using convolutional neural networks for solving ill-posed reconstruction problems, the QSM community also adapted these techniques and demonstrated that the QSM processing steps can be solved by efficient feed forward multiplications not requiring iterative optimization nor the choice of regularization parameters. Here, we review the current status of deep learning based approaches for processing QSM, highlighting limitations and potential pitfalls, and discuss the future directions the field may take to exploit the latest advances in deep learning for QSM.