PyTorchKR Papers

최신 논문

총 8,922편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2012.02234

COVID-CLNet: COVID-19 Detection with Compressive Deep Learning Approaches

One of the most serious global health threat is COVID-19 pandemic. The emphasis on improving diagnosis and increasing the diagnostic capability helps stopping its spread significantly. Therefore, to assist the radiologist or other medical professional to detect and identify the COVID-19 cases in the shortest possible time, we propose a computer-aided detection (CADe) system that uses the computed tomography (CT) scan images. This proposed boosted deep learning network (CLNet) is based on the implementation of Deep Learning (DL) networks as a complementary to the Compressive Learning (CL). We utilize our inception feature extraction technique in the measurement domain using CL to represent the data features into a new space with less dimensionality before accessing the Convolutional Neural Network. All original features have been contributed equally in the new space using a sensing matrix. Experiments performed on different compressed methods show promising results for COVID-19 detection. In addition, our novel weighted method based on different sensing matrices that used to capture boosted features demonstrates an improvement in the performance of the proposed method.

arXiv:2012.02732공식 코드 · Papers with Code

Nimble: Lightweight and Parallel GPU Task Scheduling for Deep Learning

Deep learning (DL) frameworks take advantage of GPUs to improve the speed of DL inference and training. Ideally, DL frameworks should be able to fully utilize the computation power of GPUs such that the running time depends on the amount of computation assigned to GPUs. Yet, we observe that in scheduling GPU tasks, existing DL frameworks suffer from inefficiencies such as large scheduling overhead and unnecessary serial execution. To this end, we propose Nimble, a DL execution engine that runs GPU tasks in parallel with minimal scheduling overhead. Nimble introduces a novel technique called ahead-of-time (AoT) scheduling. Here, the scheduling procedure finishes before executing the GPU kernel, thereby removing most of the scheduling overhead during run time. Furthermore, Nimble automatically parallelizes the execution of GPU tasks by exploiting multiple GPU streams in a single GPU. Evaluation on a variety of neural networks shows that compared to PyTorch, Nimble speeds up inference and training by up to 22.34$\times$ and 3.61$\times$, respectively. Moreover, Nimble outperforms state-of-the-art inference systems, TensorRT and TVM, by up to 2.81$\times$ and 1.70$\times$, respectively.

arXiv:2012.02364

Deep Learning for Medical Anomaly Detection -- A Survey

Machine learning-based medical anomaly detection is an important problem that has been extensively studied. Numerous approaches have been proposed across various medical application domains and we observe several similarities across these distinct applications. Despite this comparability, we observe a lack of structured organisation of these diverse research applications such that their advantages and limitations can be studied. The principal aim of this survey is to provide a thorough theoretical analysis of popular deep learning techniques in medical anomaly detection. In particular, we contribute a coherent and systematic review of state-of-the-art techniques, comparing and contrasting their architectural differences as well as training algorithms. Furthermore, we provide a comprehensive overview of deep model interpretation strategies that can be used to interpret model decisions. In addition, we outline the key limitations of existing deep medical anomaly detection techniques and propose key research directions for further investigation.

arXiv:2012.02282

Creativity of Deep Learning: Conceptualization and Assessment

While the potential of deep learning (DL) for automating simple tasks is already well explored, recent research has started investigating the use of deep learning for creative design, both for complete artifact creation and supporting humans in the creation process. In this paper, we use insights from computational creativity to conceptualize and assess current applications of generative deep learning in creative domains identified in a literature review. We highlight parallels between current systems and different models of human creativity as well as their shortcomings. While deep learning yields results of high value, such as high-quality images, their novelty is typically limited due to multiple reasons such as being tied to a conceptual space defined by training data. Current DL methods also do not allow for changes in the internal problem representation, and they lack the capability to identify connections across highly different domains, both of which are seen as major drivers of human creativity.

arXiv:2012.02260공식 코드 · Papers with Code

Deep Learning for Road Traffic Forecasting: Does it Make a Difference?

Deep Learning methods have been proven to be flexible to model complex phenomena. This has also been the case of Intelligent Transportation Systems (ITS), in which several areas such as vehicular perception and traffic analysis have widely embraced Deep Learning as a core modeling technology. Particularly in short-term traffic forecasting, the capability of Deep Learning to deliver good results has generated a prevalent inertia towards using Deep Learning models, without examining in depth their benefits and downsides. This paper focuses on critically analyzing the state of the art in what refers to the use of Deep Learning for this particular ITS research area. To this end, we elaborate on the findings distilled from a review of publications from recent years, based on two taxonomic criteria. A posterior critical analysis is held to formulate questions and trigger a necessary debate about the issues of Deep Learning for traffic forecasting. The study is completed with a benchmark of diverse short-term traffic forecasting methods over traffic datasets of different nature, aimed to cover a wide spectrum of possible scenarios. Our experimentation reveals that Deep Learning could not be the best modeling technique for every case, which unveils some caveats unconsidered to date that should be addressed by the community in prospective studies. These insights reveal new challenges and research opportunities in road traffic forecasting, which are enumerated and discussed thoroughly, with the intention of inspiring and guiding future research efforts in this field.

arXiv:2012.01671

ResPerfNet: Deep Residual Learning for Regressional Performance Modeling of Deep Neural Networks

The rapid advancements of computing technology facilitate the development of diverse deep learning applications. Unfortunately, the efficiency of parallel computing infrastructures varies widely with neural network models, which hinders the exploration of the design space to find high-performance neural network architectures on specific computing platforms for a given application. To address such a challenge, we propose a deep learning-based method, ResPerfNet, which trains a residual neural network with representative datasets obtained on the target platform to predict the performance for a deep neural network. Our experimental results show that ResPerfNet can accurately predict the execution time of individual neural network layers and full network models on a variety of platforms. In particular, ResPerfNet achieves 8.4% of mean absolute percentage error for LeNet, AlexNet and VGG16 on the NVIDIA GTX 1080Ti, which is substantially lower than the previously published works.

arXiv:2012.01598공식 코드 · Papers with Code

Graph Neural Networks for Improved El Niño Forecasting

Deep learning-based models have recently outperformed state-of-the-art seasonal forecasting models, such as for predicting El Ni\~no-Southern Oscillation (ENSO). However, current deep learning models are based on convolutional neural networks which are difficult to interpret and can fail to model large-scale atmospheric patterns called teleconnections. Hence, we propose the application of spatiotemporal Graph Neural Networks (GNN) to forecast ENSO at long lead times, finer granularity and improved predictive skill than current state-of-the-art methods. The explicit modeling of information flow via edges may also allow for more interpretable forecasts. Preliminary results are promising and outperform state-of-the art systems for projections 1 and 3 months ahead.

arXiv:2012.01478

Leveraging Uncertainty from Deep Learning for Trustworthy Materials Discovery Workflows

In this paper, we leverage predictive uncertainty of deep neural networks to answer challenging questions material scientists usually encounter in machine learning based materials applications workflows. First, we show that by leveraging predictive uncertainty, a user can determine the required training data set size necessary to achieve a certain classification accuracy. Next, we propose uncertainty guided decision referral to detect and refrain from making decisions on confusing samples. Finally, we show that predictive uncertainty can also be used to detect out-of-distribution test samples. We find that this scheme is accurate enough to detect a wide range of real-world shifts in data, e.g., changes in the image acquisition conditions or changes in the synthesis conditions. Using microstructure information from scanning electron microscope (SEM) images as an example use case, we show that leveraging uncertainty-aware deep learning can significantly improve the performance and dependability of classification models.

arXiv:2012.01474

Second-Order Guarantees in Federated Learning

Federated learning is a useful framework for centralized learning from distributed data under practical considerations of heterogeneity, asynchrony, and privacy. Federated architectures are frequently deployed in deep learning settings, which generally give rise to non-convex optimization problems. Nevertheless, most existing analysis are either limited to convex loss functions, or only establish first-order stationarity, despite the fact that saddle-points, which are first-order stationary, are known to pose bottlenecks in deep learning. We draw on recent results on the second-order optimality of stochastic gradient algorithms in centralized and decentralized settings, and establish second-order guarantees for a class of federated learning algorithms.

arXiv:2010.14420

SSLIDE: Sound Source Localization for Indoors based on Deep Learning

This paper presents SSLIDE, Sound Source Localization for Indoors using DEep learning, which applies deep neural networks (DNNs) with encoder-decoder structure to localize sound sources with random positions in a continuous space. The spatial features of sound signals received by each microphone are extracted and represented as likelihood surfaces for the sound source locations in each point. Our DNN consists of an encoder network followed by two decoders. The encoder obtains a compressed representation of the input likelihoods. One decoder resolves the multipath caused by reverberation, and the other decoder estimates the source location. Experiments based on both the simulated and experimental data show that our method can not only outperform multiple signal classification (MUSIC), steered response power with phase transform (SRP-PHAT), sparse Bayesian learning (SBL), and a competing convolutional neural network (CNN) approach in the reverberant environment but also achieve a good generalization performance.

arXiv:2010.09183

Modified EP MIMO Detection Algorithm with Deep Learning Parameters Selection

Expectation Propagation (EP)-based Multiple-Input Multiple-Output (MIMO) detector is regarded as a state-of-the-art MIMO detector because of its exceptional performance. However, we find that the EP MIMO detector cannot guarantee to achieve the optimal performance due to the empirical parameter selection, including initial variance and damping factors. According to the influence of the moment matching and parameter selection for the performance of the EP MIMO detector, we propose a modified EP MIMO detector (MEPD). In order to obtain the optimal initial variance and damping factors, we adopt a deep learning scheme, in which we unfold the iterative processing of MEPD to establish MEPNet for parameters training. The simulation results show that MEPD with off-line trained parameters outperforms the original one in various MIMO scenarios. Besides, the proposed MEPD with deep learning parameters selection is more robust than EPD in practical scenarios.

arXiv:2010.07052

Channel Type Recognition in Wireless Communications: A Deep Learning Approach

In this paper, we propose two novel and practical deep-learning-based algorithms to solve the wireless channel type (WCT) recognition problem. Specifically, the WCT recognition problem is recast as a classification problem in deep learning due to their similarities, where a deep neural network (DNN) is trained off-line with a diversity of typical WCTs for fifth-generation (5G) and beyond-5G wireless communications, which is then utilized to perform online WCT determination. In the first algorithm, one WCT is regarded as a single task. While in the second scheme, one WCT is jointly characterized by several independent features, each of which is treated as a task and is classified respectively by training a DNN in a multi-task-learning manner, and the final WCT is identified by the combination of those channel features. Simulation results show that the proposed algorithms can classify various WCTs instantaneously with high accuracy, result in satisfactory block error rate and throughput, and outperform a representative baseline WCT determination scheme.

arXiv:2009.13777

DeepRegularizer: Rapid Resolution Enhancement of Tomographic Imaging using Deep Learning

Optical diffraction tomography measures the three-dimensional refractive index map of a specimen and visualizes biochemical phenomena at the nanoscale in a non-destructive manner. One major drawback of optical diffraction tomography is poor axial resolution due to limited access to the three-dimensional optical transfer function. This missing cone problem has been addressed through regularization algorithms that use a priori information, such as non-negativity and sample smoothness. However, the iterative nature of these algorithms and their parameter dependency make real-time visualization impossible. In this article, we propose and experimentally demonstrate a deep neural network, which we term DeepRegularizer, that rapidly improves the resolution of a three-dimensional refractive index map. Trained with pairs of datasets (a raw refractive index tomogram and a resolution-enhanced refractive index tomogram via the iterative total variation algorithm), the three-dimensional U-net-based convolutional neural network learns a transformation between the two tomogram domains. The feasibility and generalizability of our network are demonstrated using bacterial cells and a human leukaemic cell line, and by validating the model across different samples. DeepRegularizer offers more than an order of magnitude faster regularization performance compared to the conventional iterative method. We envision that the proposed data-driven approach can bypass the high time complexity of various image reconstructions in other imaging modalities.

arXiv:2009.12168

Transient Classification in low SNR Gravitational Wave data using Deep Learning

The recent advances in Gravitational-wave astronomy have greatly accelerated the study of Multimessenger astrophysics. There is a need for the development of fast and efficient algorithms to detect non-astrophysical transients and noises due to the rate and scale at which the data is being provided by LIGO and other gravitational wave observatories. These transients and noises can interfere with the study of gravitational waves and binary mergers and induce false positives. Here, we propose the use of deep learning algorithms to detect and classify these transient signals. Traditional statistical methods are not well designed for dealing with temporal signals but supervised deep learning techniques such as RNN-LSTM and deep CNN have proven to be effective for solving problems such as time-series forecasting and time-series classification. We also use unsupervised models such as Total variation, Principal Component Analysis, Support Vector Machine, Wavelet decomposition or Random Forests for feature extraction and noise reduction and then study the results obtained by RNN-LSTM and deep CNN for classifying the transients in low-SNR signals. We compare the results obtained by the combination of various unsupervised models and supervised models. This method can be extended to real-time detection of transients and merger signals using deep-learning optimized GPU's for early prediction and study of various astronomical events. We will also explore and compare other machine learning models such as MLP, Stacked Autoencoder, Random forests, extreme learning machine, Support Vector machine and logistic regression classifier.

arXiv:2009.11075

A Deep Learning Approach for Dynamic Balance Sheet Stress Testing

In the aftermath of the financial crisis, supervisory authorities have considerably altered the mode of operation of financial stress testing. Despite these efforts, significant concerns and extensive criticism have been raised by market participants regarding the considered unrealistic methodological assumptions and simplifications. Current stress testing methodologies attempt to simulate the risks underlying a financial institution's balance sheet by using several satellite models. This renders their integration a really challenging task, leading to significant estimation errors. Moreover, advanced statistical techniques that could potentially capture the non-linear nature of adverse shocks are still ignored. This work aims to address these criticisms and shortcomings by proposing a novel approach based on recent advances in Deep Learning towards a principled method for Dynamic Balance Sheet Stress Testing. Experimental results on a newly collected financial/supervisory dataset, provide strong empirical evidence that our paradigm significantly outperforms traditional approaches; thus, it is capable of more accurately and efficiently simulating real world scenarios.

arXiv:2009.07456

Adversarial Attacks on Co-Occurrence Features for GAN Detection

Improvements in Generative Adversarial Networks (GANs) have greatly reduced the difficulty of producing new, photo-realistic images with unique semantic meaning. With this rise in ability to generate fake images comes demand to detect them. While numerous methods have been developed for this task, the majority of them remain vulnerable to adversarial attacks. In this paper, develop two novel adversarial attacks on co-occurrence based GAN detectors. These are the first attacks to be presented against such a detector. We show that our method can reduce accuracy from over 98% to less than 4%, with no knowledge of the deep learning model or weights. Furthermore, accuracy can be reduced to 0% with full knowledge of the deep learning model details.

arXiv:2009.05533

Deep Learning Interference Cancellation in Wireless Networks

With the crowding of the electromagnetic spectrum and the shrinking cell size in wireless networks, crosstalk between base stations and users is a major problem. Although hand-crafted functional blocks and coding schemes are proven effective to guarantee reliable data transfer, currently deep learning-based approaches have drawn increasing attention in the communication system modeling. In this paper, we propose a Neural Network (NN) based signal processing technique that works with traditional DSP algorithms to overcome the interference problem in realtime. This technique doesn't require any feedback protocol between the receiver and transmitter which makes it very suitable for low-latency and high data-rate applications such as autonomy and augmented reality. While there has been recent work on the use of Reinforcement Learning (RL) in the control layer to manage and control the interference, our approach is novel in the sense that it introduces a neural network for signal processing at baseband data rate and in the physical layer. We demonstrate this "Deep Interference Cancellation" technique using a convolutional LSTM autoencoder. When applied to QAM-OFDM modulated data, the network produces significant improvement in the symbol error rate (SER). We further discuss the hardware implementation including latency, power consumption, memory requirements, and chip area.

arXiv:2008.13570

Deep Learning based Spectral CT Imaging

Spectral computed tomography (CT) has attracted much attention in radiation dose reduction, metal artifacts removal, tissue quantification and material discrimination. The x-ray energy spectrum is divided into several bins, each energy-bin-specific projection has a low signal-noise-ratio (SNR) than the current-integrating counterpart, which makes image reconstruction a unique challenge. Traditional wisdom is to use prior knowledge based iterative methods. However, this kind of methods demands a great computational cost. Inspired by deep learning, here we first develop a deep learning based reconstruction method; i.e., U-net with L_p^p-norm, Total variation, Residual learning, and Anisotropic adaption (ULTRA). Specifically, we emphasize the Various Multi-scale Feature Fusion and Multichannel Filtering Enhancement with a denser connection encoding architecture for residual learning and feature fusion. To address the image deblurring problem associated with the $L_2^2$-loss, we propose a general $L_p^p$-loss, $p>0$ Furthermore, the images from different energy bins share similar structures of the same object, the regularization characterizing correlations of different energy bins is incorporated into the $L_p^p$-loss function, which helps unify the deep learning based methods with traditional compressed sensing based methods. Finally, the anisotropically weighted total variation is employed to characterize the sparsity in the spatial-spectral domain to regularize the proposed network. In particular, we validate our ULTRA networks on three large-scale spectral CT datasets, and obtain excellent results relative to the competing algorithms. In conclusion, our quantitative and qualitative results in numerical simulation and preclinical experiments demonstrate that our proposed approach is accurate, efficient and robust for high-quality spectral CT image reconstruction.

arXiv:2008.10812

Multiview Variational Deep Learning with Application to Scalable Indoor Localization

Radio channel state information (CSI) measured with many receivers is a good resource for localizing a transmit device with machine learning with a discriminative model. However, CSI localization is nontrivial when the radio map is complicated, such as in building corridors. This paper introduces a view-selective deep learning (VSDL) system for indoor localization using CSI of WiFi. The multiview training with CSI obtained from multiple groups of access points (APs) generates latent features on supervised variational deep network. This information is then applied to an additional network for dominant view classification to enhance the regression accuracy of localization. As non-informative latent features from multiple views are rejected, we can achieve a localization accuracy of 0.77 m, which outperforms by 30 $\%$ the best known accuracy in practical applications in a real building environment. To the best of our knowledge, this is the first approach to apply variational inference and to construct a scalable system for radio localization. Furthermore, our work investigates a methodology for supervised learning with multiview data where informative and non-informative views coexist.

arXiv:2007.10462공식 코드 · Papers with Code

Deep Local Volatility

Deep learning for option pricing has emerged as a novel methodology for fast computations with applications in calibration and computation of Greeks. However, many of these approaches do not enforce any no-arbitrage conditions, and the subsequent local volatility surface is never considered. In this article, we develop a deep learning approach for interpolation of European vanilla option prices which jointly yields the full surface of local volatilities. We demonstrate the modification of the loss function or the feed forward network architecture to enforce (hard constraints approach) or favor (soft constraints approach) the no-arbitrage conditions and we specify the experimental design parameters that are needed for adequate performance. A novel component is the use of the Dupire formula to enforce bounds on the local volatility associated with option prices, during the network fitting. Our methodology is benchmarked numerically on real datasets of DAX vanilla options.