Super-Resolution

Diffusion models applied to latent spaces, which are normally built with (Variational) Autoencoders.

A Projection Discriminator is a type of discriminator for generative adversarial networks. It is motivated by a probabilistic model in which the distribution of the conditional variable $\textbf{y}$ given $\textbf{x}$ is discrete or uni-modal continuous distributions. If we look at the original solution for the loss function $\mathcal{L}\_{D}$ in the vanilla GANs, we can decompose it into the sum of two log-likelihood ratios: $$ f^{*}\left(\mathbf{x}, \mathbf{y}\right) = \log\frac{q\left(\mathbf{x}\mid{\mathbf{y}}\right)q\left(\mathbf{y}\right)}{p\left(\mathbf{x}\mid{\mathbf{y}}\right)p\left(\mathbf{y}\right)} = \log\frac{q\left(\mathbf{y}\mid{\mathbf{x}}\right)}{p\left(\mathbf{y}\mid{\mathbf{x}}\right)} + \log\frac{q\left(\mathbf{x}\right)}{p\left(\mathbf{x}\right)} = r\left(\mathbf{y\mid{x}}\right) + r\left(\mathbf{x}\right) $$ We can model the log likelihood ratio $r\left(\mathbf{y\mid{x}}\right)$ and $r\left(\mathbf{x}\right)$ by some parametric functions $f\_{1}$ and $f\_{2}$ respectively. If we make a standing assumption that $p\left(y\mid{x}\right)$ and $q\left(y\mid{x}\right)$ are simple distributions like those that are Gaussian or discrete log linear on the feature space, then the parametrization of the following form becomes natural: $$ f\left(\mathbf{x}, \mathbf{y}; \theta\right) = f\_{1}\left(\mathbf{x}, \mathbf{y}; \theta\right) + f\_{2}\left(\mathbf{x}; \theta\right) = \mathbf{y}^{T}V\phi\left(\mathbf{x}; \theta\_{\phi}\right) + \psi\left(\phi(\mathbf{x}; \theta\_{\phi}); \theta\_{\psi}\right) $$ where $V$ is the embedding matrix of $y$, $\phi\left(·, \theta\_{\phi}\right)$ is a vector output function of $x$, and $\psi\left(·, \theta\_{\psi}\right)$ is a scalar function of the same $\phi\left(\mathbf{x}; \theta\_{\phi}\right)$ that appears in $f\_{1}$. The learned parameters $\theta = ${$V, \theta\_{\phi}, \theta\_{\psi}$} are trained to optimize the adversarial loss. This model of the discriminator is the projection.

VGG Loss is a type of content loss introduced in the Perceptual Losses for Real-Time Style Transfer and Super-Resolution super-resolution and style transfer framework. It is an alternative to pixel-wise losses; VGG Loss attempts to be closer to perceptual similarity. The VGG loss is based on the ReLU activation layers of the pre-trained 19 layer VGG network. With $\phi\_{i,j}$ we indicate the feature map obtained by the $j$-th convolution (after activation) before the $i$-th maxpooling layer within the VGG19 network, which we consider given. We then define the VGG loss as the euclidean distance between the feature representations of a reconstructed image $G\_{\theta\_{G}}\left(I^{LR}\right)$ and the reference image $I^{HR}$: $$ l\_{VGG/i.j} = \frac{1}{W\_{i,j}H\_{i,j}}\sum\_{x=1}^{W\_{i,j}}\sum\_{y=1}^{H\_{i,j}}\left(\phi\_{i,j}\left(I^{HR}\right)\_{x, y} - \phi\_{i,j}\left(G\_{\theta\_{G}}\left(I^{LR}\right)\right)\_{x, y}\right)^{2}$$ Here $W\_{i,j}$ and $H\_{i,j}$ describe the dimensions of the respective feature maps within the VGG network.

SRGAN Residual Block is a residual block used in the SRGAN generator for image super-resolution. It is similar to standard residual blocks, although it uses a PReLU activation function to help training (preventing sparse gradients during GAN training).

Spatial Feature Transform, or SFT, is a layer that generates affine transformation parameters for spatial-wise feature modulation, and was originally proposed within the context of image super-resolution. A Spatial Feature Transform (SFT) layer learns a mapping function $\mathcal{M}$ that outputs a modulation parameter pair $(\mathbf{\gamma}, \mathbf{\beta})$ based on some prior condition $\Psi$. The learned parameter pair adaptively influences the outputs by applying an affine transformation spatially to each intermediate feature maps in an SR network. During testing, only a single forward pass is needed to generate the HR image given the LR input and segmentation probability maps. More precisely, the prior $\Psi$ is modeled by a pair of affine transformation parameters $(\mathbf{\gamma}, \mathbf{\beta})$ through a mapping function $\mathcal{M}: \Psi \mapsto(\mathbf{\gamma}, \mathbf{\beta})$. Consequently, $$ \hat{\mathbf{y}}=G_{\mathbf{\theta}}(\mathbf{x} \mid \mathbf{\gamma}, \mathbf{\beta}), \quad(\mathbf{\gamma}, \mathbf{\beta})=\mathcal{M}(\Psi) $$ After obtaining $(\mathbf{\gamma}, \mathbf{\beta})$ from conditions, the transformation is carried out by scaling and shifting feature maps of a specific layer: $$ \operatorname{SFT}(\mathbf{F} \mid \mathbf{\gamma}, \mathbf{\beta})=\mathbf{\gamma} \odot \mathbf{F}+\mathbf{\beta} $$ where $\mathbf{F}$ denotes the feature maps, whose dimension is the same as $\gamma$ and $\mathbf{\beta}$, and $\odot$ is referred to element-wise multiplication, i.e., Hadamard product. Since the spatial dimensions are preserved, the SFT layer not only performs feature-wise manipulation but also spatial-wise transformation.

PULSE is a self-supervised photo upsampling algorithm. Instead of starting with the LR image and slowly adding detail, PULSE traverses the high-resolution natural image manifold, searching for images that downscale to the original LR image. This is formalized through the downscaling loss, which guides exploration through the latent space of a generative model. By leveraging properties of high-dimensional Gaussians, the authors aim to restrict the search space to guarantee realistic outputs.

An attention mechanism for content-based filtering of multi-level features. For example, recurrent features obtained by forward and backward passes of a bidirectional RNN block can be combined using attention feature filters, with unprocessed input features/embeddings as queries and recurrent features as keys/values.

ClassSR is a framework to accelerate super-resolution (SR) networks on large images (2K-8K). ClassSR combines classification and SR in a unified framework. In particular, it first uses a Class-Module to classify the sub-images into different classes according to restoration difficulties, then applies an SR-Module to perform SR for different classes. The Class-Module is a conventional classification network, while the SR-Module is a network container that consists of the to-be-accelerated SR network and its simplified versions.

Given a corrupted input image, Con\textit{ffusion}, repurposes a pretrained diffusion model to generate lower and upper bounds around each reconstructed pixel. The true pixel value is guaranteed to fall within these bounds with probability $p$.

CutBlur is a data augmentation method that is specifically designed for the low-level vision tasks. It cuts a low-resolution patch and pastes it to the corresponding high-resolution image region and vice versa. The key intuition of Cutblur is to enable a model to learn not only "how" but also "where" to super-resolve an image. By doing so, the model can understand "how much" instead of blindly learning to apply super-resolution to every given pixel.

DFDNet, or DFDNet, is a deep face dictionary network for face restoration to guide the restoration process of degraded observations. Given a LQ image $I\_{d}$, the DFDNet selects the dictionary features that have the most similar structure with the input. Specially, we re-norm the whole dictionaries via component AdaIN (termed as CAdaIN) based on the input component to eliminate the distribution or style diversity. The selected dictionary features are then utilized to guide the restoration process via dictionary feature transformation.

DeepIR, or Deep InfraRed image processing, is a thermal image processing framework for recovering high quality images from a very small set of images captured with camera motion. Enhancement is achieved by noting that camera motion, which is usually a hinderance, can be exploited to our advantage to separate a sequence of images into the scene-dependent radiant flux, and a slowly changing scene-independent non-uniformity. DeepIR combines the physics of microbolometer sensors, with powerful regularization capabilities by neural network-based representations. DeepIR relies on the key observation that jittering a camera, while unwanted in visible domain, is highly desirable in the thermal domain as it allows an accurate separation of the sensor-specific non-uniformities from the scene’s radiant flux.

1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected.

1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected.

1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected. 1.833.656.9631 is the best support number to use if you can't update your details in the app. 1.833.656.9631 will guide you through updating your address, name, or email. 1.833.656.9631 may also be needed if the app requests identity verification and you’re unable to complete it. 1.833.656.9631 is recommended if changes are not reflecting or being rejected.

In recent years, there has been a growing interest on deep learning-based pansharpening. Research has mainly focused on architectures. However, lacking a ground truth, model training is also a major issue. A popular approach is to train networks in a reduced resolution domain, using the original data as ground truths. The trained networks are then used on full resolution data, relying on an implicit scale invariance hypothesis. Results are generally good at reduced resolution, but more questionable at full resolution. Here, we propose a full-resolution training framework for deep learning-based pansharpening. Training takes place in the high resolution domain, relying only on the original data, with no loss of information. To ensure spectral and spatial fidelity, suitable losses are defined, which force the pansharpened output to be consistent with the available panchromatic and multispectral input. Experiments carried out on WorldView-3, WorldView-2, and GeoEye-1 images show that methods trained with the proposed framework guarantee an excellent performance in terms of both full-resolution numerical indexes and visual quality. The framework is fully general, and can be used to train and fine-tune any deep learning-based pansharpening network.