In this work we study the behavior of the forward-backward (FB) algorithm when the proximity operator is replaced by a sub-iterative procedure to approximate a Gaussian denoiser, in a Plug-and-Play (PnP) fashion. In particular, we consider both analysis and synthesis Gaussian denoisers within a dictionary framework, obtained by unrolling dual-FB iterations or FB iterations, respectively. We analyze the associated minimization problems as well as the asymptotic behavior of the resulting FB-PnP iterations. In particular, we show that the synthesis Gaussian denoising problem can be viewed as a proximity operator. For each case, analysis and synthesis, we show that the FB-PnP algorithms solve the same problem whether we use only one or an infinite number of sub-iteration to solve the denoising problem at each iteration. To this aim, we show that each "one sub-iteration" strategy within the FB-PnP can be interpreted as a primal-dual algorithm when a warm-restart strategy is used. We further present similar results when using a Moreau-Yosida smoothing of the global problem, for an arbitrary number of sub-iterations. Finally, we provide numerical simulations to illustrate our theoretical results. In particular we first consider a toy compressive sensing example, as well as an image restoration problem in a deep dictionary framework.
Large-scale networks exist in many field and play an important role in real-world dynamics. However, the networks are usually latent and expensive to detect, which becomes the main challenging for many applications and empirical analysis. Several statistical methods were proposed to infer the edges, but the complexity of algorithms make them hard to be applied for large-scale networks. In this paper, we proposed a general distributed and parallel computing framework for network reconstruction methods via compressive sensing technical, to make them feasible for inferring the super large networks in practice. Combining with the CALMS, we proposed for those estimators enjoy additional theoretical properties, such as the consistency and asymptotic normality, we prove that the approximate estimation utilizing the distributed algorithm can keep the theoretical results.
Compressive sensing (CS), acquiring and reconstructing signals below the Nyquist rate, has great potential in image and video acquisition to exploit data redundancy and greatly reduce the amount of sampled data. To further reduce the sampled data while keeping the video quality, this paper explores the temporal redundancy in video CS and proposes a block based adaptive compressive sensing framework with a sampling rate (SR) control strategy. To avoid redundant compression of non-moving regions, we first incorporate moving block detection between consecutive frames, and only transmit the measurements of moving blocks. The non-moving regions are reconstructed from the previous frame. In addition, we propose a block storage system and a dynamic threshold to achieve adaptive SR allocation to each frame based on the area of moving regions and target SR for controlling the average SR within the target SR. Finally, to reduce blocking artifacts and improve reconstruction quality, we adopt a cooperative reconstruction of the moving and non-moving blocks by referring to the measurements of the non-moving blocks from the previous frame. Extensive experiments have demonstrated that this work is able to control SR and obtain better performance than existing works.
This work addresses the fundamental linear inverse problem in compressive sensing (CS) by introducing a new type of regularizing generative prior. Our proposed method utilizes ideas from classical dictionary-based CS and, in particular, sparse Bayesian learning (SBL), to integrate a strong regularization towards sparse solutions. At the same time, by leveraging the notion of conditional Gaussianity, it also incorporates the adaptability from generative models to training data. However, unlike most state-of-the-art generative models, it is able to learn from a few compressed and noisy data samples and requires no optimization algorithm for solving the inverse problem. Additionally, similar to Dirichlet prior networks, our model parameterizes a conjugate prior enabling its application for uncertainty quantification. We support our approach theoretically through the concept of variational inference and validate it empirically using different types of compressible signals.
This work considers an uplink asynchronous massive random access scenario in which a large number of users asynchronously access a base station equipped with multiple receive antennas. The objective is to alleviate the problem of massive collision due to the limited number of orthogonal preambles of an access scheme in which user activity detection is performed. We propose a user activity detection with delay-calibration (UAD-DC) algorithm and investigate the benefits of oversampling for the estimation of continuous time delays at the receiver. The proposed algorithm iteratively estimates time delays and detects active users by noting that the collided users can be identified through accurate estimation of time delays. Due to the sporadic user activity patterns, the user activity detection problem can be formulated as a compressive sensing (CS) problem, which can be solved by a modified Turbo-CS algorithm under the consideration of correlated noise samples resulting from oversampling. A sliding-window technique is applied in the proposed algorithm to reduce the overall computational complexity. Moreover, we propose a new design of the pulse shaping filter by minimizing the Bayesian Cram\'er-Rao bound of the detection problem under the constraint of limited spectral bandwidth. Numerical results demonstrate the efficacy of the proposed algorithm in terms of the normalized mean squared error of the estimated channel, the probability of misdetection and the successful detection ratio.
Deep Neural Networks (DNNs) are well-known to act as over-parameterized deep image priors (DIP) that regularize various image inverse problems. Meanwhile, researchers also proposed extremely compact, under-parameterized image priors (e.g., deep decoder) that are strikingly competent for image restoration too, despite a loss of accuracy. These two extremes push us to think whether there exists a better solution in the middle: between over- and under-parameterized image priors, can one identify "intermediate" parameterized image priors that achieve better trade-offs between performance, efficiency, and even preserving strong transferability? Drawing inspirations from the lottery ticket hypothesis (LTH), we conjecture and study a novel "lottery image prior" (LIP) by exploiting DNN inherent sparsity, stated as: given an over-parameterized DNN-based image prior, it will contain a sparse subnetwork that can be trained in isolation, to match the original DNN's performance when being applied as a prior to various image inverse problems. Our results validate the superiority of LIPs: we can successfully locate the LIP subnetworks from over-parameterized DIPs at substantial sparsity ranges. Those LIP subnetworks significantly outperform deep decoders under comparably compact model sizes (by often fully preserving the effectiveness of their over-parameterized counterparts), and they also possess high transferability across different images as well as restoration task types. Besides, we also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior (in contrast to untrained DIP or deep decoder), and confirm its validity in this setting too. To our best knowledge, this is the first time that LTH is demonstrated to be relevant in the context of inverse problems or image priors.
Code-domain non-orthogonal multiple access (CD-NOMA) systems offer key benefits such as high spectral efficiency, low latency, high reliability, and massive connectivity. NOMA's ability to handle overloading allows multiple devices to share a single resource element (RE) for data transmission. In CD-NOMA, different users are assigned distinct codewords, which are leveraged during multi-user detection (MUD). Codebook-based NOMA systems outperform spread-sequence (SS)-based NOMA due to the coding gains provided by the codebooks. Sparse code multiple access (SCMA) and dense code multiple access (DCMA) are two prominent examples of such systems. Additionally, NOMA is seen as a crucial technology for enabling grant-free access, especially in massive machine-type communications (mMTC). One of the main challenges in deploying grant-free NOMA systems is accurately detecting both user activity and transmitted data, particularly when user activity fluctuates dynamically across the transmission frame. This paper introduces codebook-based grant-free NOMA systems modeled using a block sparsity signal structure. The joint activity and data detection (JADD) problem in these systems is formulated as group LASSO and sparse group LASSO block compressive sensing (BCS) problems. To address these, a robust prior information-aided alternating direction method of multipliers (ADMM) algorithm is proposed. Extensive numerical experiments and theoretical analysis show the efficiency of the proposed algorithm, making it a suitable solution for mMTC networks.
Radio-interferometry (RI) observes the sky at unprecedented angular resolutions, enabling the study of several far-away galactic objects such as galaxies and black holes. In RI, an array of antennas probes cosmic signals coming from the observed region of the sky. The covariance matrix of the vector gathering all these antenna measurements offers, by leveraging the Van Cittert-Zernike theorem, an incomplete and noisy Fourier sensing of the image of interest. The number of noisy Fourier measurements -- or visibilities -- scales as $\mathcal O(Q^2B)$ for $Q$ antennas and $B$ short-time integration (STI) intervals. We address the challenges posed by this vast volume of data, which is anticipated to increase significantly with the advent of large antenna arrays, by proposing a compressive sensing technique applied directly at the level of the antenna measurements. First, this paper shows that beamforming -- a common technique of dephasing antenna signals -- usually used to focus some region of the sky, is equivalent to sensing a rank-one projection (ROP) of the signal covariance matrix. We build upon our recent work arXiv:2306.12698v3 [eess.IV] to propose a compressive sensing scheme relying on random beamforming, trading the $Q^2$-dependence of the data size for a smaller number $N_{\mathrm p}$ ROPs. We provide image recovery guarantees for sparse image reconstruction. Secondly, the data size is made independent of $B$ by applying $N_{\mathrm m}$ Bernoulli modulations of the ROP vectors obtained for the STI. The resulting sample complexities, theoretically derived in a simpler case without modulations and numerically obtained in phase transition diagrams, are shown to scale as $\mathcal O(K)$ where $K$ is the image sparsity. This illustrates the potential of the approach.
Kronecker compressed sensing refers to using Kronecker product matrices as sparsifying bases and measurement matrices in compressed sensing. This work focuses on the Kronecker compressed sensing problem, encompassing three sparsity structures: $(i)$ a standard sparsity model with arbitrarily positioned nonzero entries, $(ii)$ a hierarchical sparsity model where nonzero entries are concentrated in a few blocks, each with only a subset of nonzero entries, and $(iii)$ a Kronecker-supported sparsity model where the support vector is a Kronecker product of smaller vectors. We present a hierarchal view of Kronecker compressed sensing that explicitly reveals a multiple-level sparsity pattern. This framework allows us to utilize the Kronecker structure of dictionaries and design a two-stage sparse recovery algorithm for different sparsity models. Further, we analyze the restricted isometry property of Kronecker-structured matrices under different sparsity models. Simulations show that our algorithm offers comparable recovery performance to state-of-the-art methods while significantly reducing runtime.
The ability to estimate 3D movements of users over edge computing-enabled networks, such as 5G/6G networks, is a key enabler for the new era of extended reality (XR) and Metaverse applications. Recent advancements in deep learning have shown advantages over optimization techniques for estimating 3D human poses given spare measurements from sensor signals, i.e., inertial measurement unit (IMU) sensors attached to the XR devices. However, the existing works lack applicability to wireless systems, where transmitting the IMU signals over noisy wireless networks poses significant challenges. Furthermore, the potential redundancy of the IMU signals has not been considered, resulting in highly redundant transmissions. In this work, we propose a novel approach for redundancy removal and lightweight transmission of IMU signals over noisy wireless environments. Our approach utilizes a random Gaussian matrix to transform the original signal into a lower-dimensional space. By leveraging the compressive sensing theory, we have proved that the designed Gaussian matrix can project the signal into a lower-dimensional space and preserve the Set-Restricted Eigenvalue condition, subject to a power transmission constraint. Furthermore, we develop a deep generative model at the receiver to recover the original IMU signals from noisy compressed data, thus enabling the creation of 3D human body movements at the receiver for XR and Metaverse applications. Simulation results on a real-world IMU dataset show that our framework can achieve highly accurate 3D human poses of the user using only $82\%$ of the measurements from the original signals. This is comparable to an optimization-based approach, i.e., Lasso, but is an order of magnitude faster.
Video Snapshot Compressive Imaging (SCI) uses a low-speed 2D camera to capture high-speed scenes as snapshot compressed measurements, followed by a reconstruction algorithm to retrieve the high-speed video frames. The fast evolving mobile devices and existing high-performance video SCI reconstruction algorithms motivate us to develop mobile reconstruction methods for real-world applications. Yet, it is still challenging to deploy previous reconstruction algorithms on mobile devices due to the complex inference process, let alone real-time mobile reconstruction. To the best of our knowledge, there is no video SCI reconstruction model designed to run on the mobile devices. Towards this end, in this paper, we present an effective approach for video SCI reconstruction, dubbed MobileSCI, which can run at real-time speed on the mobile devices for the first time. Specifically, we first build a U-shaped 2D convolution-based architecture, which is much more efficient and mobile-friendly than previous state-of-the-art reconstruction methods. Besides, an efficient feature mixing block, based on the channel splitting and shuffling mechanisms, is introduced as a novel bottleneck block of our proposed MobileSCI to alleviate the computational burden. Finally, a customized knowledge distillation strategy is utilized to further improve the reconstruction quality. Extensive results on both simulated and real data show that our proposed MobileSCI can achieve superior reconstruction quality with high efficiency on the mobile devices. Particularly, we can reconstruct a 256 X 256 X 8 snapshot compressed measurement with real-time performance (about 35 FPS) on an iPhone 15. Code is available at https://github.com/mcao92/MobileSCI.
One of the most promising solutions for uncertainty quantification in high-dimensional statistics is the debiased LASSO that relies on unconstrained $\ell_1$-minimization. The initial works focused on real Gaussian designs as a toy model for this problem. However, in medical imaging applications, such as compressive sensing for MRI, the measurement system is represented by a (subsampled) complex Fourier matrix. The purpose of this work is to extend the method to the MRI case in order to construct confidence intervals for each pixel of an MR image. We show that a sufficient amount of data is $n \gtrsim \max\{ s_0\log^2 s_0\log p, s_0 \log^2 p \}$.
We investigate the problem of recovering a structured sparse signal from a linear observation model with an uncertain dynamic grid in the sensing matrix. The state-of-the-art expectation maximization based compressed sensing (EM-CS) methods, such as turbo compressed sensing (Turbo-CS) and turbo variational Bayesian inference (Turbo-VBI), have a relatively slow convergence speed due to the double-loop iterations between the E-step and M-step. Moreover, each inner iteration in the E-step involves a high-dimensional matrix inverse in general, which is unacceptable for problems with large signal dimensions or real-time calculation requirements. Although there are some attempts to avoid the high-dimensional matrix inverse by majorization minimization, the convergence speed and accuracy are often sacrificed. To better address this problem, we propose an alternating estimation framework based on a novel subspace constrained VBI (SC-VBI) method, in which the high-dimensional matrix inverse is replaced by a low-dimensional subspace constrained matrix inverse (with the dimension equal to the sparsity level). We further prove the convergence of the SC-VBI to a stationary solution of the Kullback-Leibler divergence minimization problem. Simulations demonstrate that the proposed SC-VBI algorithm can achieve a much better tradeoff between complexity per iteration, convergence speed, and performance compared to the state-of-the-art algorithms.
Dynamic metasurface antennas (DMAs) represent a novel transceiver array architecture for extremely large-scale (XL) communications, offering the advantages of reduced power consumption and lower hardware costs compared to conventional arrays. This paper focuses on near-field channel estimation for XL-DMAs. We begin by analyzing the near-field characteristics of uniform planar arrays (UPAs) and introducing the Oblong Approx. model. This model decouples elevation-azimuth (EL-AZ) parameters for XL-DMAs, providing an effective means to characterize the near-field effect. It offers simpler mathematical expressions than the second-order Taylor expansion model, all while maintaining negligible model errors for oblong-shaped arrays. Building on the Oblong Approx. model, we propose an EL-AZ-decoupled estimation framework that involves near- and far-field parameter estimation for AZ/EL and EL/AZ directions, respectively. The former is formulated as a distributed compressive sensing problem, addressed using the proposed off-grid distributed orthogonal least squares algorithm, while the latter involves a straightforward parallelizable search. Crucially, we illustrate the viability of decoupled EL-AZ estimation for near-field UPAs, exhibiting commendable performance and linear complexity correlated with the number of metasurface elements. Moreover, we design an measurement matrix optimization method with the Lorentzian constraint on DMAs and highlight the estimation performance degradation resulting from this constraint.
We address the challenge of zeroth-order online convex optimization where the objective function's gradient exhibits sparsity, indicating that only a small number of dimensions possess non-zero gradients. Our aim is to leverage this sparsity to obtain useful estimates of the objective function's gradient even when the only information available is a limited number of function samples. Our motivation stems from the optimization of large-scale queueing networks that process time-sensitive jobs. Here, a job must be processed by potentially many queues in sequence to produce an output, and the service time at any queue is a function of the resources allocated to that queue. Since resources are costly, the end-to-end latency for jobs must be balanced with the overall cost of the resources used. While the number of queues is substantial, the latency function primarily reacts to resource changes in only a few, rendering the gradient sparse. We tackle this problem by introducing the Compressive Online Gradient Optimization framework which allows compressive sensing methods previously applied to stochastic optimization to achieve regret bounds with an optimal dependence on the time horizon without the full problem dimension appearing in the bound. For specific algorithms, we reduce the samples required per gradient estimate to scale with the gradient's sparsity factor rather than its full dimensionality. Numerical simulations and real-world microservices benchmarks demonstrate CONGO's superiority over gradient descent approaches that do not account for sparsity.
High-speed railway (HSR) communications are pivotal for ensuring rail safety, operations, maintenance, and delivering passenger information services. The high speed of trains creates rapidly time-varying wireless channels, increases the signaling overhead, and reduces the system throughput, making it difficult to meet the growing and stringent needs of HSR applications. In this article, we explore artificial intelligence (AI)-based beam-level and cell-level mobility management suitable for HSR communications. Particularly, we propose a compressed spatial multi-beam measurements scheme via compressive sensing for beam-level mobility management in HSR communications. In comparison to traditional down-sampling spatial beam measurements, this method leads to improved spatial-temporal beam prediction accuracy with the same measurement overhead. Moreover, we propose a novel AI-based proactive handover scheme to predict handover events and reduce radio link failure (RLF) rates in HSR communications. Compared with the traditional event A3-based handover mechanism, the proposed approach significantly reduces the RLF rates which saves 50% beam measurement overhead.
In frequency-division duplex (FDD) massive multiple-input multiple-output (MIMO) systems, the growing number of base station antennas leads to prohibitive feedback overhead for downlink channel state information (CSI). To address this challenge, state-of-the-art (SOTA) fully data-driven deep learning (DL)-based CSI feedback schemes have been proposed. However, the high computational complexity and memory requirements of these methods hinder their practical deployment on resource-constrained devices like mobile phones. To solve the problem, we propose a model-driven DL-based CSI feedback approach by integrating the wisdom of compressive sensing and learning to optimize (L2O). Specifically, only a linear learnable projection is adopted at the encoder side to compress the CSI matrix, thereby significantly cutting down the user-side complexity and memory expenditure. On the other hand, the decoder incorporates two specially designed components, i.e., a learnable sparse transformation and an element-wise L2O reconstruction module. The former is developed to learn a sparse basis for CSI within the angular domain, which explores channel sparsity effectively. The latter shares the same long short term memory (LSTM) network across all elements of the optimization variable, eliminating the retraining cost when problem scale changes. Simulation results show that the proposed method achieves a comparable performance with the SOTA CSI feedback scheme but with much-reduced complexity, and enables multiple-rate feedback.
Since most inverse problems arising in scientific and engineering applications are ill-posed, prior information about the solution space is incorporated, typically through regularization, to establish a well-posed problem with a unique solution. Often, this prior information is an assumed statistical distribution of the desired inverse problem solution. Recently, due to the unprecedented success of generative adversarial networks (GANs), the generative network from a GAN has been implemented as the prior information in imaging inverse problems. In this paper, we devise a novel iterative algorithm to solve inverse problems in imaging where a dual-structured prior is imposed by combining a GAN prior with the compound Gaussian (CG) class of distributions. A rigorous computational theory for the convergence of the proposed iterative algorithm, which is based upon the alternating direction method of multipliers, is established. Furthermore, elaborate empirical results for the proposed iterative algorithm are presented. By jointly exploiting the powerful CG and GAN classes of image priors, we find, in compressive sensing and tomographic imaging problems, our proposed algorithm outperforms and provides improved generalizability over competitive prior art approaches while avoiding performance saturation issues in previous GAN prior-based methods.
In order to extract governing equations from time-series data, various approaches are proposed. Among those, sparse identification of nonlinear dynamics (SINDy) stands out as a successful method capable of modeling governing equations with a minimal number of terms, utilizing the principles of compressive sensing. This feature, which relies on a small number of terms, is crucial for interpretability. The effectiveness of SINDy hinges on the choice of candidate functions within its dictionary to extract governing equations of dynamical systems. A larger dictionary allows for more terms, enhancing the quality of approximations. However, the computational complexity scales with dictionary size, rendering SINDy less suitable for high-dimensional datasets, even though it has been successfully applied to low-dimensional datasets. To address this challenge, we introduce iterative SINDy in this paper, where the dictionary undergoes expansion and compression through iterations. We also conduct an analysis of the convergence properties of iterative SINDy. Simulation results validate that iterative SINDy can achieve nearly identical performance to SINDy, while significantly reducing computational complexity. Notably, iterative SINDy demonstrates effectiveness with high-dimensional time-series data without incurring the prohibitively high computational cost associated with SINDy.
Sparse-aspect multi-baseline Synthetic Aperture Radar (SAR) three-dimensional (3D) tomography is a crucial remote sensing technique. Compared to full-aspect observation, it needs only a few observation aspects to achieve a sufficiently clear 3D scene reconstruction, providing a cost-effective alternative. In the past, compressive sensing (CS) was the mainstream approach for sparse 3D SAR imaging. Recently, deep learning (DL) revolutionizes this field through its powerful data-driven representation capabilities and efficient inference characteristics. However, existing DL methods primarily depend on high-resolution radar images for supervising the training of deep neural networks (DNNs). This unimodal approach precludes the incorporation of complementary information from other data sources, thereby limiting potential improvements in imaging performance. In this paper, we propose a Cross-Modal 3D-SAR Reconstruction Network (CMAR-Net) that enhances 3D SAR imaging by fusing heterogeneous information. Leveraging cross-modal supervision from 2D optical images and error transfer guaranteed by differentiable rendering, CMAR-Net achieves efficient training and reconstructs highly sparse-aspect multi-baseline SAR image into visually structured and accurate 3D images, particularly for vehicle targets. Extensive experiments on simulated and real-world datasets demonstrate that CMAR-Net significantly outperforms state-of-the-art sparse reconstruction algorithms based on CS and DL, with average improvements of 75.83% in PSNR and 47.85% in SSIM. Furthermore, our method eliminates the need for time-consuming full-aperture data preprocessing and relies solely on computer-rendered optical images, significantly reducing dataset construction costs. This work highlights the potential of cross-modal learning for multi-baseline SAR 3D imaging and introduces a novel framework for radar imaging research.