We demonstrate stable real-time operation of a software-defined, GPU-based receiver over a metropolitan network. Massive parallelization is exploited for implementing direct-detection and coherent Kramers-Kronig detection in real time at 2 and 1 GBaud, respectively.
Designing hardware accelerators for deep neural networks (DNNs) has been much desired. Nonetheless, most of these existing accelerators are built for either convolutional neural networks (CNNs) or recurrent neural networks (RNNs). Recently, the Transformer model is replacing the RNN in the natural language processing (NLP) area. However, because of intensive matrix computations and complicated data flow being involved, the hardware design for the Transformer model has never been reported. In this paper, we propose the first hardware accelerator for two key components, i.e., the multi-head attention (MHA) ResBlock and the position-wise feed-forward network (FFN) ResBlock, which are the two most complex layers in the Transformer. Firstly, an efficient method is introduced to partition the huge matrices in the Transformer, allowing the two ResBlocks to share most of the hardware resources. Secondly, the computation flow is well designed to ensure the high hardware utilization of the systolic array, which is the biggest module in our design. Thirdly, complicated nonlinear functions are highly optimized to further reduce the hardware complexity and also the latency of the entire system. Our design is coded using hardware description language (HDL) and evaluated on a Xilinx FPGA. Compared with the implementation on GPU with the same setting, the proposed design demonstrates a speed-up of 14.6x in the MHA ResBlock, and 3.4x in the FFN ResBlock, respectively. Therefore, this work lays a good foundation for building efficient hardware accelerators for multiple Transformer networks.
Synthetic Aperture Radar (SAR) is a critical sensing technology that is notably independent of the sensor-to-target distance and has numerous cross-cutting applications, e.g., target recognition, mapping, surveillance, oceanography, geology, forestry (biomass, deforestation), disaster monitoring (volcano eruptions, oil spills, flooding), and infrastructure tracking (urban growth, structure mapping). SAR uses a high-power antenna to illuminate target locations with electromagnetic radiation, e.g., 10GHz radio waves, and illuminated surface backscatter is sensed by the antenna which is then used to generate images of structures. Real SAR data is difficult and costly to produce and, for research, lacks a reliable source ground truth. This article proposes a open source SAR simulator to compute phase histories for arbitrary 3D scenes using newly available ray-tracing hardware made available commercially through the NVIDIA's RTX graphics cards series. The OptiX GPU ray tracing library for NVIDIA GPUs is used to calculate SAR phase histories at unprecedented computational speeds. The simulation results are validated against existing SAR simulation code for spotlight SAR illumination of point targets. The computational performance of this approach provides orders of magnitude speed increases over CPU simulation. An additional order of magnitude of GPU acceleration when simulations are run on RTX GPUs which include hardware specifically to accelerate OptiX ray tracing. The article describes the OptiX simulator structure, processing framework and calculations that afford execution on massively parallel GPU computation device. The shortcoming of the OptiX library's restriction to single precision float representation is discussed and modifications of sensitive calculations are proposed to reduce truncation error thereby increasing the simulation accuracy under this constraint.
In recent years, x-ray speckle tracking techniques have emerged as viable tools for wavefront metrology and sample imaging applications. These methods are based on the measurement of near-field images. Thanks to the simple experimental set-up, high angular sensitivity and compatibility with low coherence sources these methods have been actively developed for use with synchrotron and laboratory light sources. Not only do speckle-tracking techniques give the potential for high resolution imaging, but they also provide rapid and robust characterisation of aberrations of x-ray optical elements, focal spot profiles and the sample position and transmission properties. In order to realise these capabilities, we require software implementations that are equally rapid and robust. To address this need, a software suite has been developed for the "ptychographic x-ray speckle tracking technique" -- an x-ray speckle based method suitable for highly divergent wavefields. The software suite is written in Python 3, with an OpenCL back end for GPU and multi-CPU core processing. It is accessible as a Python module, through the command line or through a graphical user interface and is available as source code under version 3 or later of the GNU General Public License.
With the rapidly-developing high-speed wireless communications, the 60 GHz millimeter-wave frequency range and radio-over-fiber systems have been investigated as a promising solution to deliver mm-wave signals. Neural networks have been studied to improve the mm-wave RoF system performances at the receiver side by suppressing linear and nonlinear impairments. However, previous neural network studies in mm-wave RoF systems focus on the off-line implementation with high-end GPUs , which is not practical for low power-consumption, low-cost and limited computation platform applications. To solve this issue, we investigate neural network hardware accelerator implementations using the field programmable gate array (FPGA), taking advantage of the low power consumption, parallel computation capability, and reconfigurablity features of FPGA. Convolutional neural network (CNN) and binary convolutional neural network (BCNN) hardware accelerators are demonstrated. In addition, to satisfy the low-latency requirement in mm-wave RoF systems and to enable the use of low-cost compact FPGA devices, a novel inner parallel optimization method is proposed. Compared with the embedded processor (ARM Cortex A9) execution latency, the CNN/BCNN FPGA-based hardware accelerator reduces their latency by over 92%. Compared with non-optimized FPGA implementations, the proposed optimization method reduces the processing latency by over 44% for CNN and BCNN. Compared with the GPU implementation, the latency of CNN implementation with the proposed optimization method is reduced by 85.49%, while the power consumption is reduced by 86.91%. Although the latency of BCNN implementation with the proposed optimization method is larger compared with the GPU implementation, the power consumption is reduced by 86.14%. The FPGA-based neural network hardware accelerators provide a promising solution for mm-wave RoF systems.
This work presents and evaluates a novel input parameterization method which improves the tractability of model predictive control (MPC) for high degree of freedom (DoF) robots. Experimental results demonstrate that by parameterizing the input trajectory more than three quarters of the optimization variables used in traditional MPC can be eliminated with practically no effect on system performance. This parameterization also leads to trajectories which are more conservative, producing less overshoot in underdamped systems with modeling error. In this paper we present two MPC solution methods that make use of this parameterization. The first uses a convex solver, and the second makes use of parallel computing on a graphics processing unit (GPU). We show that both approaches drastically reduce solve times for large DoF, long horizon MPC problems allowing solutions at real-time rates. Through simulation and hardware experiments, we show that the parameterized convex solver MPC has faster solve times than traditional MPC for high DoF cases while still achieving similar performance. For the GPU-based MPC solution method, we use an evolutionary algorithm and that we call Evolutionary MPC (EMPC). EMPC is shown to have even faster solve times for high DoF systems. Solve times for EMPC are shown to decrease even further through the use of a more powerful GPU. This suggests that parallelized MPC methods will become even more advantageous with the improvement and prevalence of GPU technology.
Software defined radio is a widely accepted paradigm for design of reconfigurable modems. The continuing march of Moore's law makes real-time signal processing on general purpose processors feasible for a large set of waveforms. Data rates in the low Mbps can be processed on low-power ARM processors, and much higher data rates can be supported on large x86 processors. The advantages of all-software development (vs. FPGA/DSP/GPU) are compelling: much wider pool of talent, lower development time and cost, and easier maintenance and porting. However, very high-rate systems (above 100 Mbps) are still firmly in the domain of custom and semi-custom hardware (mostly FPGAs). In this paper we describe an architecture and testbed for an SDR that can be easily scaled to support over 3 GHz of bandwidth and data rate up to 10 Gbps. The paper covers a novel technique to parallelize typically serial algorithms for phase and symbol tracking, followed by a discussion of data distribution for a massively parallel architecture. We provide a brief description of a mixed-signal front end and conclude with measurement results. To the best of the author's knowledge, the system described in this paper is an order of magnitude faster than any prior published result.
This paper presents a Gated Recurrent Unit (GRU) based recurrent neural network (RNN) accelerator called EdgeDRNN designed for portable edge computing. EdgeDRNN adopts the spiking neural network inspired delta network algorithm to exploit temporal sparsity in RNNs. It reduces off-chip memory access by a factor of up to 10x with tolerable accuracy loss. Experimental results on a 10 million parameter 2-layer GRU-RNN, with weights stored in DRAM, show that EdgeDRNN computes them in under 0.5 ms. With 2.42 W wall plug power on an entry level USB powered FPGA board, it achieves latency comparable with a 92 W Nvidia 1080 GPU. It outperforms NVIDIA Jetson Nano, Jetson TX2 and Intel Neural Compute Stick 2 in latency by 6X. For a batch size of 1, EdgeDRNN achieves a mean effective throughput of 20.2 GOp/s and a wall plug power efficiency that is over 4X higher than all other platforms.
Uncovering the heterogeneity of cell populations is a long-standing goal in fields ranging from antimicrobial resistance to cancer research. Emerging technology platforms such as droplet microfluidics hold the promise to decipher cellular heterogeneity at ultra-high-throughput. However, there is a lack of methods able to rapidly identify and isolate single cells or 3D cell cultures. Here we demonstrate that deep neural networks can accurately classify single droplet images in real-time based on the presence and number of micro-objects including single mammalian cells and multicellular spheroids. This approach also enables the identification of specific objects within mixtures of objects of different types and sizes. The training sets for the neural networks consisted of several hundred images manually picked and augmented to up to thousands of images per training class. Training required less than 10 minutes using a single GPU, and yielded accuracies of over 90% for single mammalian cell identification. Crucially, the same model could be used to classify different types of objects such as polystyrene spheres, polyacrylamide beads and MCF-7 cells. We applied the developed method to the selection of 3D cell cultures generated with Hek293FT cells encapsulated in agarose gel beads, highlighting the potential of the technology for the selection of objects with high diversity of visual appearance. The real-time sorting of single droplets was in-line with droplet generation and occurred at rates up to 40 per second independently of image size up to 480 x 480 pixels. The presented microfluidic device also enabled storage of sorted droplets to allow for downstream analyses.
A 3D ISAR security imaging experiment at 24GHz is demonstrated with a sparse MIMO array. The MIMO array is an 8Tx/16Rx linear array to achieve real-aperture imaging along the vertical dimension. It is time-switching multiplexed with a low-cost FMCW transceiver working at 22GHz-26GHz. A calibration procedure is proposed to calibrate the channel imbalance across the MIMO array. The experiment is conducted on human moving on a cart, where we take advantage of the linear motion of human to form inverse synthetic aperture along the horizontal dimension. To track the motion of human, a 3D depth camera is used as an auxiliary sensor to capture the rough position of target to aid ISAR imaging. The back projection imaging algorithm is implemented on GPU for quasi-real-time operation. Finally, experiments are conducted with real human with concealed objects and a preliminary automatic object recognition algorithm based on convolutional neural networks are developed and evaluated on real data.
We propose a heterogeneous simultaneous graphical dynamic linear model (H-SGDLM), which extends the standard SGDLM framework to incorporate a heterogeneous autoregressive realised volatility (HAR-RV) model. This novel approach creates a GPU-scalable multivariate volatility estimator, which decomposes multiple time series into economically-meaningful variables to explain the endogenous and exogenous factors driving the underlying variability. This unique decomposition goes beyond the classic one step ahead prediction; indeed, we investigate inferences up to one month into the future using stocks, FX futures and ETF futures, demonstrating its superior performance according to accuracy of large moves, longer-term prediction and consistency over time.
The NEURON simulator has been developed over the past three decades and is widely used by neuroscientists to model the electrical activity of neuronal networks. Large network simulation projects using NEURON have supercomputer allocations that individually measure in the millions of core hours. Supercomputer centers are transitioning to next generation architectures and the work accomplished per core hour for these simulations could be improved by an order of magnitude if NEURON was able to better utilize those new hardware capabilities. In order to adapt NEURON to evolving computer architectures, the compute engine of the NEURON simulator has been extracted and has been optimized as a library called CoreNEURON. This paper presents the design, implementation and optimizations of CoreNEURON. We describe how CoreNEURON can be used as a library with NEURON and then compare performance of different network models on multiple architectures including IBM BlueGene/Q, Intel Skylake, Intel MIC and NVIDIA GPU. We show how CoreNEURON can simulate existing NEURON network models with 4-7x less memory usage and 2-7x less execution time while maintaining binary result compatibility with NEURON.
While Model Based Iterative Reconstruction (MBIR) of CT scans has been shown to have better image quality than Filtered Back Projection (FBP), its use has been limited by its high computational cost. More recently, deep convolutional neural networks (CNN) have shown great promise in both denoising and reconstruction applications. In this research, we propose a fast reconstruction algorithm, which we call Deep Learning MBIR (DL-MBIR), for approximating MBIR using a deep residual neural network. The DL-MBIR method is trained to produce reconstructions that approximate true MBIR images using a 16 layer residual convolutional neural network implemented on multiple GPUs using Google Tensorflow. In addition, we propose 2D, 2.5D and 3D variations on the DL-MBIR method and show that the 2.5D method achieves similar quality to the fully 3D method, but with reduced computational cost.
Deep neural network (DNN) architectures, such as convolutional neural networks (CNN), involve heavy computation and require hardware, such as CPU, GPU, and AI accelerators, to provide the massive computing power. With the many varieties of AI hardware prevailing on the market, it is often hard to decide which one is the best to use. Thus, benchmarking AI hardware effectively becomes important and is of great help to select and optimize AI hardware. Unfortunately, there are few AI benchmarks available in both academia and industry. Examples are BenchNN[1], DeepBench[2], and Dawn Bench[3], which are usually a collection of typical real DNN applications. While these benchmarks provide performance comparison across different AI hardware, they suffer from a number of drawbacks. First, they cannot adapt to the emerging changes of DNN algorithms and are fixed once selected. Second, they contain tens to hundreds of applications and take very long time to finish running. Third, they are mainly selected from open sources, which are restricted by copyright and are not representable to proprietary applications. In this work, a synthetic benchmarks framework is firstly proposed to address the above drawbacks of AI benchmarks. Instead of pre-selecting a set of open-sourced benchmarks and running all of them, the synthetic approach generates only a one or few benchmarks that best represent a broad range of applications using profiled workload characteristics data of these applications. Thus, it can adapt to emerging changes of new DNN algorithms by re-profiling new applications and updating itself, greatly reduce benchmark count and running time, and strongly represent DNN applications of interests. The generated benchmarks are called AI Matrix, serving as a performance benchmarks matching the statistical workload characteristics of a combination of applications of interests.
In our former works we have made serious efforts to improve the performance of medical image analysis methods with using ensemble-based systems. In this paper, we present a novel hardware-based solution for the efficient adoption of our complex, fusion-based approaches for real-time applications. Even though most of the image processing problems and the increasing amount of data have high-performance computing(HPC) demand, there is still a lack of corresponding dedicated HPC solutions for several medical tasks. To widen this bottleneck we have developed a Hybrid Small Size high performance computing Resource (abbreviated by HuSSaR) which efficiently alloys CPU and GPU technologies, mobile and has an own cooling system to support easy mobility and wide applicability. Besides a proper technical description, we include several practical examples from the clinical data processing domain in this work. For more details see also: https://arato.inf.unideb.hu/kovacs.laszlo/research_hybridmicrohpc.html
In this paper, we propose a cross layer energy efficient resource allocation and remote radio head (RRH) selection algorithm for heterogeneous traffic in power domain - non-orthogonal multiple access (PD-NOMA) based heterogeneous cloud radio access networks (H-CRANs). The main aim is to maximize the EE of the elastic users subject to the average delay constraint of the streaming users and the constraints, RRH selection, subcarrier, transmit power and successive interference cancellation. The considered optimization problem is non-convex, NP-hard and intractable. To solve this problem, we transform the fractional objective function into a subtractive form. Then, we utilize successive convex approximation approach. Moreover, in order to increase the processing speed, we introduce a framework for accelerating the successive convex approximation for low complexity with the Lagrangian method on graphics processing unit. Furthermore, in order to show the optimality gap of the proposed successive convex approximation approach, we solve the proposed optimization problem by applying an optimal method based on the monotonic optimization. Studying different scenarios show that by using both PD-NOMA technique and H-CRAN, the system energy efficiency is improved.
The amount of unstructured text-based data is growing every day. Querying, clustering, and classifying this big data requires similarity computations across large sets of documents. Whereas low-complexity similarity metrics are available, attention has been shifting towards more complex methods that achieve a higher accuracy. In particular, the Word Mover's Distance (WMD) method proposed by Kusner et al. is a promising new approach, but its time complexity grows cubically with the number of unique words in the documents. The Relaxed Word Mover's Distance (RWMD) method, again proposed by Kusner et al., reduces the time complexity from qubic to quadratic and results in a limited loss in accuracy compared with WMD. Our work contributes a low-complexity implementation of the RWMD that reduces the average time complexity to linear when operating on large sets of documents. Our linear-complexity RWMD implementation, henceforth referred to as LC-RWMD, maps well onto GPUs and can be efficiently distributed across a cluster of GPUs. Our experiments on real-life datasets demonstrate 1) a performance improvement of two orders of magnitude with respect to our GPU-based distributed implementation of the quadratic RWMD, and 2) a performance improvement of three to four orders of magnitude with respect to our distributed WMD implementation that uses GPU-based RWMD for pruning.
The realized stochastic volatility (RSV) model that utilizes the realized volatility as additional information has been proposed to infer volatility of financial time series. We consider the Bayesian inference of the RSV model by the Hybrid Monte Carlo (HMC) algorithm. The HMC algorithm can be parallelized and thus performed on the GPU for speedup. The GPU code is developed with CUDA Fortran. We compare the computational time in performing the HMC algorithm on GPU (GTX 760) and CPU (Intel i7-4770 3.4GHz) and find that the GPU can be up to 17 times faster than the CPU. We also code the program with OpenACC and find that appropriate coding can achieve the similar speedup with CUDA Fortran.
BarraCUDA is a C program which uses the BWA algorithm in parallel with nVidia CUDA to align short next generation DNA sequences against a reference genome. The genetically improved (GI) code is up to three times faster on short paired end reads from The 1000 Genomes Project and 60percent more accurate on a short BioPlanet.com GCAT alignment benchmark. GPGPU Barracuda running on a single K80 Tesla GPU can align short paired end nextgen sequences up to ten times faster than bwa on a 12 core CPU.
Nowadays, one practical limitation of deep neural network (DNN) is its high degree of specialization to a single task or domain (e.g., one visual domain). It motivates researchers to develop algorithms that can adapt DNN model to multiple domains sequentially, while still performing well on the past domains, which is known as multi-domain learning. Almost all conventional methods only focus on improving accuracy with minimal parameter update, while ignoring high computing and memory cost during training, which makes it difficult to deploy multi-domain learning into more and more widely used resource-limited edge devices, like mobile phones, IoT, embedded systems, etc. We observe that large memory used for activation storage is the bottleneck that largely limits the training time and cost on edge devices. To reduce training memory usage, while keeping the domain adaption accuracy performance, we propose Dynamic Additive Attention Adaption ($DA^3$), a novel memory-efficient on-device multi-domain learning method. $DA^3$ learns a novel additive attention adaptor module, while freezing the weights of the pre-trained backbone model for each domain. Differentiating from prior works, such module not only mitigates activation memory buffering for reducing memory usage during training but also serves as a dynamic gating mechanism to reduce the computation cost for fast inference. We validate $DA^3$ on multiple datasets against state-of-the-art methods, which shows great improvement in both accuracy and training time. Moreover, we deployed $DA^3$ into the popular NIVDIA Jetson Nano edge GPU, where the measured experimental results show our proposed $DA^3$ reduces the on-device training memory consumption by 19-37x, and training time by 2x, in comparison to the baseline methods (e.g., standard fine-tuning, Parallel and Series Res. adaptor, and Piggyback).