Cycle Consistency Loss is a type of loss used for generative adversarial networks that performs unpaired image-to-image translation. It was introduced with the CycleGAN architecture. For two domains $X$ and $Y$, we want to learn a mapping $G : X \rightarrow Y$ and $F: Y \rightarrow X$. We want to enforce the intuition that these mappings should be reverses of each other and that both mappings should be bijections. Cycle Consistency Loss encourages $F\left(G\left(x\right)\right) \approx x$ and $G\left(F\left(y\right)\right) \approx y$. It reduces the space of possible mapping functions by enforcing forward and backwards consistency: $$ \mathcal{L}\_{cyc}\left(G, F\right) = \mathbb{E}\_{x \sim p\_{data}\left(x\right)}\left[||F\left(G\left(x\right)\right) - x||\_{1}\right] + \mathbb{E}\_{y \sim p\_{data}\left(y\right)}\left[||G\left(F\left(y\right)\right) - y||\_{1}\right] $$
Colorization Transformer is a probabilistic colorization model composed only of axial self-attention blocks. The main advantages of these blocks are the ability to capture a global receptive field with only two layers and $\mathcal{O}(D\sqrt{D})$ instead of $\text{O}(D^{2})$ complexity. In order to enable colorization of high-resolution grayscale images, the task is decomposed into three simpler sequential subtasks: coarse low resolution autoregressive colorization, parallel color and spatial super-resolution. For coarse low resolution colorization, a conditional variant of Axial Transformer is applied. The authors leverage the semi-parallel sampling mechanism of Axial Transformers. Finally, fast parallel deterministic upsampling models are employed to super-resolve the coarsely colorized image into the final high resolution output.