The morphological structure of left ventricle segmented from cardiac magnetic resonance images can be used to calculate key clinical parameters, and it is of great significance to the accurate and efficient diagnosis of cardiovascular diseases. Compared with traditional methods, the segmentation algorithms based on fully convolutional neural network greatly improve the accuracy of semantic segmentation. For the problem of left ventricular segmentation, a new fully convolutional neural network structure named MS-FCN is proposed in this paper. The MS-FCN network employs a multi-scale pooling module to ensure that the network maximises the feature extraction ability and uses a dense connectivity decoder to refine the boundaries of the object. Based on the Sunnybrook cine-MR dataset provided by the MICCAI 2009 challenge, numerical experiments demonstrate that our proposed model has obtained state-of-the-art segmentation results: the Dice score of our method reaches 0.93 on the endocardium, and 0.96 on the epicardium.
We consider a bistatic configuration with a stationary transmitter transmitting unknown waveforms of opportunity and a moving receiver, and present a Deep Learning (DL) framework for passive synthetic aperture radar (SAR) imaging. Existing passive radar methods require two or more antennas which are either spatially separated or colocated with sufficient directivity to estimate the underlying waveform prior to imaging. Our approach to passive radar only requires a single receiver, hence reduces cost and increases versatility. We approach DL from an optimization perspective and formulate image reconstruction as a machine learning task. By unfolding the iterations of a proximal gradient descent algorithm, we construct a deep recurrent neural network (RNN) that is parameterized by transmitted waveforms. We cascade the RNN structure with a decoder stage to form a recurrent-auto encoder architecture. We then utilize backpropagation to learn transmitted waveforms by training the network in an unsupervised manner using SAR measurements. The highly non-convex problem of backpropagation is guided to a feasible solution over the parameter space by initializing the network with the known components of the SAR forward model. Moreover, prior information regarding the waveform structure is incorporated during initialization and backpropagation. We demonstrate the effectiveness of the DL-based approach through extensive numerical simulations that show focused, high contrast imagery using a single receiver antenna at realistic SNR levels.
In this paper, we propose a null-space-based preemptive scheduling framework for cross-objective optimization to always guarantee robust URLLC performance, while extracting the maximum possible eMBB capacity. The proposed scheduler perpetually grants incoming URLLC traffic a higher priority for instant scheduling. In case that radio resources are not immediately schedulable, proposed scheduler forcibly enforces an artificial spatial user separation, for the URLLC traffic to get instantly scheduled over shared resources with ongoing eMBB transmissions. A pre-defined reference spatial subspace is constructed for which scheduler instantly picks the active eMBB user whose precoder is the closest possible. Then, it projects the eMBB precoder on-the-go onto the reference subspace, in order for its paired URLLC user to orient its decoder matrix into one possible null space of the reference subspace. Hence, a robust decoding ability is always preserved at the URLLC user, while cross-maximizing the ergodic capacity. Compared to the state-of-the-art proposals from industry and academia, proposed scheduler shows extreme URLLC latency robustness with significantly improved overall spectral efficiency. Analytical analysis and extensive system level simulations are presented to support paper conclusions.
We consider the problem of designing a low-complexity decoder for antipodal uniquely decodable (UD) /errorless code sets for overloaded synchronous code-division multiple access (CDMA) systems, where the number of signals Kamax is the largest known for the given code length L. In our complexity analysis, we illustrate that compared to maximum-likelihood (ML) decoder, which has an exponential computational complexity for even moderate code lengths, the proposed decoder has a quasi-quadratic computational complexity. Simulation results in terms of bit-error-rate (BER) demonstrate that the performance of the proposed decoder has only a 1-2 dB degradation in signal-to-noise ratio (SNR) at a BER of 10^-3 when compared to ML. Moreover, we derive the proof of the minimum Manhattan distance of such UD codes and we provide the proofs for the propositions; these proofs constitute the foundation of the formal proof for the maximum number users Kamax for L=8 .
We describe block oriented multidimensional pulse position modulation and its resilience against impulsive noise. The modulation implements the encoder and part of the decoder of the BBC algorithm. We tested the modulation on circuits that send and detect a pulse based signal in the presence of impulsive noise. We measured the packet error rate vs. signal to noise ratio and we compared it with published error rates for OFDM. We found an error rate of 2 x 10^(-5) at a signal to noise ratio of 16 dB without forward error correction and a data rate of 64 kbit /sec.
We investigate a turbo soft detector based on the expectation propagation (EP) algorithm for large-scale multiple-input multiple-output (MIMO) systems. Optimal detection in MIMO systems becomes computationally unfeasible for high-order modulations and/or large number of antennas. In this situation, the linear minimum mean square error (LMMSE) exhibits a low-complexity with a good performance, however far from optimal. To improve the performance, the EP algorithm can be used. In this paper, we review previous EP-based detectors and enhance their estimation in terms of complexity and performance. Specifically, we improve the convergence of the self-iterated EP stage by replacing the uniform prior by a non-uniform one, which better characterizes the information returned by the decoder once the turbo procedure starts. We also review the EP parameters to avoid instabilities when using high-order modulations and to reduce the computational complexity. Simulation results illustrate the robustness and enhanced performance of this novel detector in comparison with previous approaches found in the literature. Results also show that the proposed detector is robust in the presence of imperfect channel state information (CSI).
Recognizing emotions using few attribute dimensions such as arousal, valence and dominance provides the flexibility to effectively represent complex range of emotional behaviors. Conventional methods to learn these emotional descriptors primarily focus on separate models to recognize each of these attributes. Recent work has shown that learning these attributes together regularizes the models, leading to better feature representations. This study explores new forms of regularization by adding unsupervised auxiliary tasks to reconstruct hidden layer representations. This auxiliary task requires the denoising of hidden representations at every layer of an auto-encoder. The framework relies on ladder networks that utilize skip connections between encoder and decoder layers to learn powerful representations of emotional dimensions. The results show that ladder networks improve the performance of the system compared to baselines that individually learn each attribute, and conventional denoising autoencoders. Furthermore, the unsupervised auxiliary tasks have promising potential to be used in a semi-supervised setting, where few labeled sentences are available.
An original expectation propagation (EP) based message passing framework is introduced, wherein transmitted symbols are considered to belong to the multivariate white Gaussian distribution family. This approach allows deriving a novel class of single-tap frequency domain (FD) receivers with a quasi-linear computational complexity in block length, thanks to Fast-Fourier transform (FFT) based implementation. This framework is exposed in detail, through the design of a novel double-loop single-carrier frequency domain equalizer (SC-FDE), where self-iterations of the equalizer with the demapper, and turbo iterations with the decoder, provide numerous combinations for the performance and complexity trade-off. Furthermore, the flexibility of this framework is illustrated with the derivation of an overlap FDE, used for time-varying channel equalization, among others, and with the design of a FD multiple-input multiple-output (MIMO) detector, used for spatial multiplexing. Through these different receiver design problems, this framework is shown to improve the mitigation of inter-symbol, inter-block and multi-antenna interferences, compared to alternative single-tap FD structures of previous works. Thanks to finite-length and asymptotic analysis, supported by numerical results, the improvement brought by the proposed structures is assessed, and then completed by also accounting for computational costs.
In the popular video coding trend, the encoder has the task to exploit both spatial and temporal redundancies present in the video sequence, which is a complex procedure. As a result almost all video encoders have five to ten times more complexity than their decoders. In a video compression process, one of the main tasks at the encoder side is motion estimation which is to extract the temporal correlation between frames. Distributed video coding (DVC) proposed the idea that can lead to low complexity encoders and higher complexity decoders. DVC is a new paradigm in video compression based on the information theoretic ideas of Slepian-Wolf and Wyner-Ziv theorems. Wyner-Ziv coding is naturally robust against transmission errors and can be used for joint source and channel coding. Side Information is one of the key components of the Wyner-Ziv decoder. Better side information generation will result in better functionality of Wyner-Ziv coder. In this paper we proposed a new method that can generate side information with a better quality and thus better compression. We have used HVS (human visual system) based image quality metrics as our quality criterion. The motion estimation we used in the decoder is modified due to these metrics such that we could obtain finer side information. The motion compensation is optimized for perceptual quality metrics and leads to better side information generation compared to con- ventional MSE (mean squared error) or SAD (sum of absolute difference) based motion compensation currently used in the literature. Better motion compensation means better compression.
As the first error correction codes provably achieving the symmetric capacity of binary-input discrete memory-less channels (B-DMCs), polar codes have been recently chosen by 3GPP for eMBB control channel. Among existing algorithms, CRC-aided successive cancellation list (CA-SCL) decoding is favorable due to its good performance, where CRC is placed at the end of the decoding and helps to eliminate the invalid candidates before final selection. However, the good performance is obtained with a complexity increase that is linear in list size $L$. In this paper, the tailored CRC-aided SCL (TCA-SCL) decoding is proposed to balance performance and complexity. Analysis on how to choose the proper CRC for a given segment is proposed with the help of \emph{virtual transform} and \emph{virtual length}. For further performance improvement, hybrid automatic repeat request (HARQ) scheme is incorporated. Numerical results have shown that, with the similar complexity as the state-of-the-art, the proposed TCA-SCL and HARQ-TCA-SCL schemes achieve $0.1$ dB and $0.25$ dB performance gain at frame error rate $\textrm{FER}=10^{-2}$, respectively. Finally, an efficient TCA-SCL decoder is implemented with FPGA demonstrating its advantages over CA-SCL decoder.
We consider the problem of estimating a Gaussian random walk from a lossy compression of its decimated version. Hence, the encoder operates on the decimated random walk, and the decoder estimates the original random walk from its encoded version under a mean squared error (MSE) criterion. It is well-known that the minimal distortion in this problem is attained by an estimate-and-compress (EC) source coding strategy, in which the encoder first estimates the original random walk and then compresses this estimate subject to the bit constraint. In this work, we derive a closed-form expression for this minimal distortion as a function of the bitrate and the decimation factor. Next, we consider a compress-and-estimate (CE) source coding scheme, in which the encoder first compresses the decimated sequence subject to an MSE criterion (with respect to the decimated sequence), and the original random walk is estimated only at the decoder. We evaluate the distortion under CE in a closed form and show that there exists a nonzero gap between the distortion under the two schemes. This difference in performance illustrates the importance of having the decimation factor at the encoder.
The conventional grant-based network relies on the handshaking between base station and active users to achieve dynamic multi-user scheduling, which may cost large signaling overheads as well as system latency. To address those problems, the grant-free receiver design is considered in this paper based on sparse code multiple access (SCMA), one of the promising air interface technologies for 5G wireless networks. With the presence of unknown multipath fading, the proposed receiver blindly performs joint channel estimation and data decoding without knowing the user activity in the network. Based on the framework of belief propagation (BP), we formulate a messagepassing receiver for uplink SCMA that performs joint estimation iteratively. However, the direct application of BP for the multivariable detection problem is complex. Motivated by the idea of approximate inference, we use expectation propagation to project the intractable distributions into Gaussian families such that a linear complexity decoder is obtained.Simulation results show that the proposed receiver can detect active users in the network with a high accuracy and can achieve an improved bit-error-rate performance compared with existing methods.
This manuscript has been submitted to Transactions on Communications on September 7, 2017; revised on January 10, 2018 and March 27, 2018; and accepted on April 25, 2018 We propose a novel filter-type equalizer to improve the solution of the linear minimum-mean squared-error (LMMSE) turbo equalizer, with computational complexity constrained to be quadratic in the filter length. When high-order modulations and/or large memory channels are used the optimal BCJR equalizer is unavailable, due to its computational complexity. In this scenario, the filter-type LMMSE turbo equalization exhibits a good performance compared to other approximations. In this paper, we show that this solution can be significantly improved by using expectation propagation (EP) in the estimation of the a posteriori probabilities. First, it yields a more accurate estimation of the extrinsic distribution to be sent to the channel decoder. Second, compared to other solutions based on EP the computational complexity of the proposed solution is constrained to be quadratic in the length of the finite impulse response (FIR). In addition, we review previous EP-based turbo equalization implementations. Instead of considering default uniform priors we exploit the outputs of the decoder. Some simulation results are included to show that this new EP-based filter remarkably outperforms the turbo approach of previous versions of the EP algorithm and also improves the LMMSE solution, with and without turbo equalization.
Advances in CMOS technology have made high resolution image sensors possible. These image sensor pose significant challenges in terms of the amount of raw data generated, energy efficiency and frame rate. This paper presents a new design methodology for an imaging system and a simplified novel image sensor pixel design to be used in such system so that Compressed Sensing (CS) technique can be implemented easily at the sensor level. This results in significant energy savings as it not only cuts the raw data rate but also reduces transistor count per pixel, decreases pixel size, increases fill factor, simplifies ADC, JPEG encoder and JPEG decoder design and decreases wiring as well as address decoder size by half. Thus CS has the potential to increase the resolution of image sensors for a given technology and die size while significantly decreasing the power consumption and design complexity. We show that it has potential to reduce power consumption by about 23%-65%.
Gene expression levels carry information about signals that have functional significance for the organism. Using the gap gene network in the fruit fly embryo as an example, we show how this information can be decoded, building a dictionary that translates expression levels into a map of implied positions. The optimal decoder makes use of graded variations in absolute expression level, resulting in positional estimates that are precise to ~1% of the embryo's length. We test this optimal decoder by analyzing gap gene expression in embryos lacking some of the primary maternal inputs to the network. The resulting maps are distorted, and these distortions predict, with no free parameters, the positions of expression stripes for the pair-rule genes in the mutant embryos.
Highly connected recurrent neural networks often produce chaotic dynamics, meaning their precise activity is sensitive to small perturbations. What are the consequences for how such networks encode streams of temporal stimuli? On the one hand, chaos is a strong source of randomness, suggesting that small changes in stimuli will be obscured by intrinsically generated variability. On the other hand, recent work shows that the type of chaos that occurs in spiking networks can have a surprisingly low-dimensional structure, suggesting that there may be "room" for fine stimulus features to be precisely resolved. Here we show that strongly chaotic networks produce patterned spikes that reliably encode time-dependent stimuli: using a decoder sensitive to spike times on timescales of 10's of ms, one can easily distinguish responses to very similar inputs. Moreover, recurrence serves to distribute signals throughout chaotic networks so that small groups of cells can encode substantial information about signals arriving elsewhere. A conclusion is that the presence of strong chaos in recurrent networks does not prohibit precise stimulus encoding.
Grid cells in the entorhinal cortex encode the position of an animal in its environment using spatially periodic tuning curves of varying periodicity. Recent experiments established that these cells are functionally organized in discrete modules with uniform grid spacing. Here we develop a theory for efficient coding of position, which takes into account the temporal statistics of the animal's motion. The theory predicts a sharp decrease of module population sizes with grid spacing, in agreement with the trends seen in the experimental data. We identify a simple scheme for readout of the grid cell code by neural circuitry, that can match in accuracy the optimal Bayesian decoder of the spikes. This readout scheme requires persistence over varying timescales, ranging from ~1ms to ~1s, depending on the grid cell module. Our results suggest that the brain employs an efficient representation of position which takes advantage of the spatiotemporal statistics of the encoded variable, in similarity to the principles that govern early sensory coding.
Current Cyber-Physical Systems (CPSs) are sophisticated, complex, and equipped with networked sensors and actuators. As such, they have become further exposed to cyber-attacks. Recent catastrophic events have demonstrated that standard, human-based management of anomaly detection in complex systems is not efficient enough and have underlined the significance of automated detection, intelligent and rapid response. Nevertheless, existing anomaly detection frameworks usually are not capable of dealing with the dynamic and complicated nature of the CPSs. In this study, we introduce an unsupervised framework for anomaly detection based on an Attention-based Spatio-Temporal Autoencoder. In particular, we first construct statistical correlation matrices to characterize the system status across different time steps. Next, a 2D convolutional encoder is employed to encode the patterns of the correlation matrices, whereas an Attention-based Convolutional LSTM Encoder-Decoder (ConvLSTM-ED) is used to capture the temporal dependencies. More precisely, we introduce an input attention mechanism to adaptively select the most significant input features at each time step. Finally, the 2D convolutional decoder reconstructs the correlation matrices. The differences between the reconstructed correlation matrices and the original ones are used as indicators of anomalies. Extensive experimental analysis on data collected from all six stages of Secure Water Treatment (SWaT) testbed, a scaled-down version of a real-world industrial water treatment plant, demonstrates that the proposed model outperforms the state-of-the-art baseline techniques.
Removing undesired reflection from an image captured through a glass surface is a very challenging problem with many practical application scenarios. For improving reflection removal, cascaded deep models have been usually adopted to estimate the transmission in a progressive manner. However, most existing methods are still limited in exploiting the result in prior stage for guiding transmission estimation. In this paper, we present a novel two-stage network with reflection-aware guidance (RAGNet) for single image reflection removal (SIRR). To be specific, the reflection layer is firstly estimated due to that it generally is much simpler and is relatively easier to estimate. Reflectionaware guidance (RAG) module is then elaborated for better exploiting the estimated reflection in predicting transmission layer. By incorporating feature maps from the estimated reflection and observation, RAG can be used (i) to mitigate the effect of reflection from the observation, and (ii) to generate mask in partial convolution for mitigating the effect of deviating from linear combination hypothesis. A dedicated mask loss is further presented for reconciling the contributions of encoder and decoder features. Experiments on five commonly used datasets demonstrate the quantitative and qualitative superiority of our RAGNet in comparison to the state-of-the-art SIRR methods. The source code and pre-trained model are available at https://github.com/liyucs/RAGNet.
Commonsense generation aims at generating plausible everyday scenario description based on a set of provided concepts. Digging the relationship of concepts from scratch is non-trivial, therefore, we retrieve prototypes from external knowledge to assist the understanding of the scenario for better description generation. We integrate two additional modules, namely position indicator and scaling module, into the pretrained encoder-decoder model for prototype modeling to enhance the knowledge injection procedure. We conduct experiment on CommonGen benchmark, and experimental results show that our method significantly improves the performance on all the metrics.