PyTorchKR Papers

최신 논문

총 8,922편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1807.11573

State-of-the-art and gaps for deep learning on limited training data in remote sensing

Deep learning usually requires big data, with respect to both volume and variety. However, most remote sensing applications only have limited training data, of which a small subset is labeled. Herein, we review three state-of-the-art approaches in deep learning to combat this challenge. The first topic is transfer learning, in which some aspects of one domain, e.g., features, are transferred to another domain. The next is unsupervised learning, e.g., autoencoders, which operate on unlabeled data. The last is generative adversarial networks, which can generate realistic looking data that can fool the likes of both a deep learning network and human. The aim of this article is to raise awareness of this dilemma, to direct the reader to existing work and to highlight current gaps that need solving.

arXiv:1807.10570

Embedded Implementation of a Deep Learning Smile Detector

In this paper we study the real time deployment of deep learning algorithms in low resource computational environments. As the use case, we compare the accuracy and speed of neural networks for smile detection using different neural network architectures and their system level implementation on NVidia Jetson embedded platform. We also propose an asynchronous multithreading scheme for parallelizing the pipeline. Within this framework, we experimentally compare thirteen widely used network topologies. The experiments show that low complexity architectures can achieve almost equal performance as larger ones, with a fraction of computation required.

arXiv:1807.09571

Deep Learning Detection Networks in MIMO Decode-Forward Relay Channels

In this paper, we consider signal detection algorithms in a multiple-input multiple-output (MIMO) decode-forward (DF) relay channel with one source, one relay, and one destination. The existing suboptimal near maximum likelihood (NML) detector and the NML with two-level pair-wise error probability (NMLw2PEP) detector achieve excellent performance with instantaneous channel state information (CSI) of the source-relay (SR) link and with statistical CSI of the SR link, respectively. However, the NML detectors require an exponentially increasing complexity as the number of transmit antennas increases. Using deep learning algorithms, NML-based detection networks (NMLDNs) are proposed with and without the CSI of the SR link at the destination. The NMLDNs detect signals in changing channels after a single training using a large number of randomly distributed channels. The detection networks require much lower detection complexity than the exhaustive search NML detectors while exhibiting good performance. To evaluate the performance, we introduce semidefinite relaxation detectors with polynomial complexity based on the NML detectors. Additionally, new linear detectors based on the zero gradient of the NML metrics are proposed. Applying various detection algorithms at the relay (DetR) and detection algorithms at the destination (DetD), we present some DetR-DetD methods in MIMO DF relay channels. An appropriate DetR-DetD method can be employed according to the required error probability and detection complexity. The complexity analysis and simulation results validate the arguments of this paper.

arXiv:1807.09351

Deep learning the high variability and randomness inside multimode fibres

Multimode fibres (MMF) are remarkable high-capacity information channels owing to the large number of transmitting fibre modes, and have recently attracted significant renewed interest in applications such as optical communication, imaging, and optical trapping. At the same time, the optical transmitting modes inside MMFs are highly sensitive to external perturbations and environmental changes, resulting in MMF transmission channels being highly variable and random. This largely limits the practical application of MMFs and hinders the full exploitation of their information capacity. Despite great research efforts made to overcome the high variability and randomness inside MMFs, any geometric change to the MMF leads to completely different transmission matrices, which unavoidably fails at the information recovery. Here, we show the successful binary image transmission using deep learning through a single MMF, which is stationary or subject to dynamic shape variations. We found that a single convolutional neural network has excellent generalisation capability with various MMF transmission states. This deep neural network can be trained by multiple MMF transmission states to accurately predict unknown information at the other end of the MMF at any of these states, without knowing which state is present. Our results demonstrate that deep learning is a promising solution to address the variability and randomness challenge of MMF based information channels. This deep-learning approach is the starting point of developing future high-capacity MMF optical systems and devices, and is applicable to optical systems concerning other diffusing media.

arXiv:1807.04813

Optimal Physical Preprocessing for Example-Based Super-Resolution

In example-based super-resolution, the function relating low-resolution images to their high-resolution counterparts is learned from a given dataset. This data-driven approach to solving the inverse problem of increasing image resolution has been implemented with deep learning algorithms. In this work, we explore modifying the imaging hardware in order to collect more informative low-resolution images for better ultimate high-resolution image reconstruction. We show that this "physical preprocessing" allows for improved image reconstruction with deep learning in Fourier ptychographic microscopy. Fourier ptychographic microscopy is a technique allowing for both high resolution and high field-of-view at the cost of temporal resolution. In Fourier ptychographic microscopy, variable illumination patterns are used to collect multiple low-resolution images. These low-resolution images are then computationally combined to create an image with resolution exceeding that of any single image from the microscope. We use deep learning to jointly optimize the illumination pattern with the post-processing reconstruction algorithm for a given sample type, allowing for single-shot imaging with both high resolution and high field-of-view. We demonstrate, with simulated data, that the joint optimization yields improved image reconstruction as compared with sole optimization of the post-processing reconstruction algorithm.

arXiv:1807.03120

Towards Radiologist-Level Accurate Deep Learning System for Pulmonary Screening

In this work, we propose advanced pneumonia and Tuberculosis grading system for X-ray images. The proposed system is a very deep fully convolutional classification network with online augmentation that outputs confidence values for diseases prevalence. Its a fully automated system capable of disease feature understanding without any offline preprocessing step or manual feature extraction. We have achieved state- of-the- art performance on the public databases such as ChestXray-14, Mendeley, Shenzhen Hospital X-ray and Belarus X-ray set.

arXiv:1807.02164

V-CNN: When Convolutional Neural Network encounters Data Visualization

In recent years, deep learning poses a deep technical revolution in almost every field and attracts great attentions from industry and academia. Especially, the convolutional neural network (CNN), one representative model of deep learning, achieves great successes in computer vision and natural language processing. However, simply or blindly applying CNN to the other fields results in lower training effects or makes it quite difficult to adjust the model parameters. In this poster, we propose a general methodology named V-CNN by introducing data visualizing for CNN. V-CNN introduces a data visualization model prior to CNN modeling to make sure the data after processing is fit for the features of images as well as CNN modeling. We apply V-CNN to the network intrusion detection problem based on a famous practical dataset: AWID. Simulation results confirm V-CNN significantly outperforms other studies and the recall rate of each invasion category is more than 99.8%.

arXiv:1806.10029

Low Photon Count Phase Retrieval Using Deep Learning

Imaging systems' performance at low light intensity is affected by shot noise, which becomes increasingly strong as the power of the light source decreases. In this paper we experimentally demonstrate the use of deep neural networks to recover objects illuminated with weak light and demonstrate better performance than with the classical Gerchberg-Saxton phase retrieval algorithm for equivalent signal over noise ratio. Prior knowledge about the object is implicitly contained in the training data set and feature detection is possible for a signal over noise ratio close to one. We apply this principle to a phase retrieval problem and show successful recovery of the object's most salient features with as little as one photon per detector pixel on average in the illumination beam. We also show that the phase reconstruction is significantly improved by training the neural network with an initial estimate of the object, as opposed as training it with the raw intensity measurement.

arXiv:1806.07810

Identifying viruses from metagenomic data by deep learning

The recent development of metagenomic sequencing makes it possible to sequence microbial genomes including viruses in an environmental sample. Identifying viral sequences from metagenomic data is critical for downstream virus analyses. The existing reference-based and gene homology-based methods are not efficient in identifying unknown viruses or short viral sequences. Here we have developed a reference-free and alignment-free machine learning method, DeepVirFinder, for predicting viral sequences in metagenomic data using deep learning techniques. DeepVirFinder was trained based on a large number of viral sequences discovered before May 2015. Evaluated on the sequences after that date, DeepVirFinder outperformed the state-of-the-art method VirFinder at all contig lengths. Enlarging the training data by adding millions of purified viral sequences from environmental metavirome samples significantly improves the accuracy for predicting under-represented viruses. Applying DeepVirFinder to real human gut metagenomic samples from patients with colorectal carcinoma (CRC) identified 51,138 viral sequences belonging to 175 bins. Ten bins were associated with the cancer status, indicating their potential use for non-invasive diagnosis of CRC. In summary, DeepVirFinder greatly improved the precision and recall rates of viral identification, and it will significantly accelerate the discovery rate of viruses.

arXiv:1804.04826

On Deep Learning-based Massive MIMO Indoor User Localization

We examine the usability of deep neural networks for multiple-input multiple-output (MIMO) user positioning solely based on the orthogonal frequency division multiplex (OFDM) complex channel coefficients. In contrast to other indoor positioning systems (IPSs), the proposed method does not require any additional piloting overhead or any other changes in the communications system itself as it is deployed on top of an existing OFDM MIMO system. Supported by actual measurements, we are mainly interested in the more challenging non-line of sight (NLoS) scenario. However, gradient descent optimization is known to require a large amount of data-points for training, i.e., the required database would be too large when compared to conventional methods. Thus, we propose a twostep training procedure, with training on simulated line of sight (LoS) data in the first step, and finetuning on measured NLoS positions in the second step. This turns out to reduce the required measured training positions and thus, reduces the effort for data acquisition.

arXiv:1803.09848

Epileptic Seizure Detection: A Deep Learning Approach

Epilepsy is the second most common brain disorder after migraine. Automatic detection of epileptic seizures can considerably improve the patients' quality of life. Current Electroencephalogram (EEG)-based seizure detection systems encounter many challenges in real-life situations. The EEGs are non-stationary signals and seizure patterns vary across patients and recording sessions. Moreover, EEG data are prone to numerous noise types that negatively affect the detection accuracy of epileptic seizures. To address these challenges, we introduce the use of a deep learning-based approach that automatically learns the discriminative EEG features of epileptic seizures. Specifically, to reveal the correlation between successive data samples, the time-series EEG data are first segmented into a sequence of non-overlapping epochs. Second, Long Short-Term Memory (LSTM) network is used to learn the high-level representations of the normal and the seizure EEG patterns. Third, these representations are fed into Softmax function for training and classification. The results on a well-known benchmark clinical dataset demonstrate the superiority of the proposed approach over the existing state-of-the-art methods. Furthermore, our approach is shown to be robust in noisy and real-life conditions. Compared to current methods that are quite sensitive to noise, the proposed method maintains its high detection performance in the presence of common EEG artifacts (muscle activities and eye-blinking) as well as white noise.

arXiv:1802.04479

Phased Microphone Array for Sound Source Localization with Deep Learning

To phased microphone array for sound source localization, algorithm with both high computational efficiency and high precision is a persistent pursuit. In this paper convolutional neural network (CNN) a kind of deep learning is preliminarily applied as a new algorithm. At high frequency CNN can reconstruct the sound localizations with excellent spatial resolution as good as DAMAS, within a very short time as short as conventional beamforming. This exciting result means that CNN perfectly finds source distribution directly from cross-spectral matrix without given propagation function in advance, and thus CNN deserves to be further explored as a new algorithm.

arXiv:1801.07130

Computational Protein Design with Deep Learning Neural Networks

Computational protein design has a wide variety of applications. Despite its remarkable success, designing a protein for a given structure and function is still a challenging task. On the other hand, the number of solved protein structures is rapidly increasing while the number of unique protein folds has reached a steady number, suggesting more structural information is being accumulated on each fold. Deep learning neural network is a powerful method to learn such big data set and has shown superior performance in many machine learning fields. In this study, we applied the deep learning neural network approach to computational protein design for predicting the probability of 20 natural amino acids on each residue in a protein. A large set of protein structures was collected and a multi-layer neural network was constructed. A number of structural properties were extracted as input features and the best network achieved an accuracy of 38.3%. Using the network output as residue type restraints was able to improve the average sequence identity in designing three natural proteins using Rosetta. Moreover, the predictions from our network show ~3% higher sequence identity than a previous method. Results from this study may benefit further development of computational protein design methods.

arXiv:1801.03406

DeepSeek: Content Based Image Search & Retrieval

Most of the internet today is composed of digital media that includes videos and images. With pixels becoming the currency in which most transactions happen on the internet, it is becoming increasingly important to have a way of browsing through this ocean of information with relative ease. YouTube has 400 hours of video uploaded every minute and many million images are browsed on Instagram, Facebook, etc. Inspired by recent advances in the field of deep learning and success that it has gained on various problems like image captioning and, machine translation , word2vec , skip thoughts, etc, we present DeepSeek a natural language processing based deep learning model that allows users to enter a description of the kind of images that they want to search, and in response the system retrieves all the images that semantically and contextually relate to the query. Two approaches are described in the following sections.

arXiv:1706.06302

Deep Learning in (and of) Agent-Based Models: A Prospectus

A very timely issue for economic agent-based models (ABMs) is their empirical estimation. This paper describes a line of research that could resolve the issue by using machine learning techniques, using multi-layer artificial neural networks (ANNs), or so called Deep Nets. The seminal contribution by Hinton et al. (2006) introduced a fast and efficient training algorithm called Deep Learning, and there have been major breakthroughs in machine learning ever since. Economics has not yet benefited from these developments, and therefore we believe that now is the right time to apply Deep Learning and multi-layered neural networks to agent-based models in economics.

arXiv:1705.05368

DeepRT: deep learning for peptide retention time prediction in proteomics

Accurate predictions of peptide retention times (RT) in liquid chromatography have many applications in mass spectrometry-based proteomics. Herein, we present DeepRT, a deep learning based software for peptide retention time prediction. DeepRT automatically learns features directly from the peptide sequences using the deep convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) model, which eliminates the need to use hand-crafted features or rules. After the feature learning, principal component analysis (PCA) was used for dimensionality reduction, then three conventional machine learning methods were utilized to perform modeling. Two published datasets were used to evaluate the performance of DeepRT and we demonstrate that DeepRT greatly outperforms previous state-of-the-art approaches ELUDE and GPTime.

arXiv:1705.04600

Concussion classification via deep learning using whole-brain white matter fiber strains

Developing an accurate and reliable injury predictor is central to the biomechanical studies of traumatic brain injury. State-of-the-art efforts continue to rely on empirical, scalar metrics based on kinematics or model-estimated tissue responses explicitly pre-defined in a specific brain region of interest. They could suffer from loss of information. A single training dataset has also been used to evaluate performance but without cross-validation. In this study, we developed a deep learning approach for concussion classification using implicit features of the entire voxel-wise white matter fiber strains. Using reconstructed American National Football League (NFL) injury cases, leave-one-out cross-validation was employed to objectively compare injury prediction performances against two baseline machine learning classifiers (support vector machine (SVM) and random forest (RF)) and four scalar metrics via univariate logistic regression (Brain Injury Criterion (BrIC), cumulative strain damage measure of the whole brain (CSDM-WB) and the corpus callosum (CSDM-CC), and peak fiber strain in the CC). Feature-based deep learning and machine learning classifiers consistently outperformed all scalar injury metrics across all performance categories in cross-validation (e.g., average accuracy of 0.844 vs. 0.746, and average area under the receiver operating curve (AUC) of 0.873 vs. 0.769, respectively, based on the testing dataset). Nevertheless, deep learning achieved the best cross-validation accuracy, sensitivity, and AUC (e.g., accuracy of 0.862 vs. 0.828 and 0.842 for SVM and RF, respectively). These findings demonstrate the superior performances of deep learning in concussion prediction, and suggest its promise for future applications in biomechanical investigations of traumatic brain injury.

arXiv:1610.05189

ProQ3D: Improved model quality assessments using Deep Learning

Summary: Protein quality assessment is a long-standing problem in bioinformatics. For more than a decade we have developed state-of-art predictors by carefully selecting and optimising inputs to a machine learning method. The correlation has increased from 0.60 in ProQ to 0.81 in ProQ2 and 0.85 in ProQ3 mainly by adding a large set of carefully tuned descriptions of a protein. Here, we show that a substantial improvement can be obtained using exactly the same inputs as in ProQ2 or ProQ3 but replacing the support vector machine by a deep neural network. This improves the Pearson correlation to 0.90 (0.85 using ProQ2 input features). Availability: ProQ3D is freely available both as a webserver and a stand-alone program at http://proq3.bioinfo.se/

arXiv:1607.02470

Deep Learning for Mortgage Risk

We develop a deep learning model of multi-period mortgage risk and use it to analyze an unprecedented dataset of origination and monthly performance records for over 120 million mortgages originated across the US between 1995 and 2014. Our estimators of term structures of conditional probabilities of prepayment, foreclosure and various states of delinquency incorporate the dynamics of a large number of loan-specific as well as macroeconomic variables down to the zip-code level. The estimators uncover the highly nonlinear nature of the relationship between the variables and borrower behavior, especially prepayment. They also highlight the effects of local economic conditions on borrower behavior. State unemployment has the greatest explanatory power among all variables, offering strong evidence of the tight connection between housing finance markets and the macroeconomy. The sensitivity of a borrower to changes in unemployment strongly depends upon current unemployment. It also significantly varies across the entire borrower population, which highlights the interaction of unemployment and many other variables. These findings have important implications for mortgage-backed security investors, rating agencies, and housing finance policymakers.

arXiv:1601.07754

Deep Learning Based Semantic Video Indexing and Retrieval

We share the implementation details and testing results for video retrieval system based exclusively on features extracted by convolutional neural networks. We show that deep learned features might serve as universal signature for semantic content of video useful in many search and retrieval tasks. We further show that graph-based storage structure for video index allows to efficiently retrieving the content with complicated spatial and temporal search queries.