기법

The Exponential Linear Squashing Activation Function, or ELiSH, is an activation function used for neural networks. It shares common properties with Swish, being made up of an ELU and a Sigmoid: $$f\left(x\right) = \frac{x}{1+e^{-x}} \text{ if } x \geq 0 $$ $$f\left(x\right) = \frac{e^{x} - 1}{1+e^{-x}} \text{ if } x < 0 $$ The Sigmoid part of ELiSH improves information flow, while the linear parts solve issues of vanishing gradients.

ERNIE-GEN is a multi-flow sequence to sequence pre-training and fine-tuning framework which bridges the discrepancy between training and inference with an infilling generation mechanism and a noise-aware generation method. To make generation closer to human writing patterns, this framework introduces a span-by-span generation flow that trains the model to predict semantically-complete spans consecutively rather than predicting word by word. Unlike existing pre-training methods, ERNIE-GEN incorporates multi-granularity target sampling to construct pre-training data, which enhances the correlation between encoder and decoder.

ESPNetv2 is a convolutional neural network that utilises group point-wise and depth-wise dilated separable convolutions to learn representations from a large effective receptive field with fewer FLOPs and parameters.

These breaches led to fraudulent transactions, unauthorized 1ー844ー314ー6480 withdrawals, and exposure of sensitive personal data. Plaintiffs alleged that1ー844ー314ー6480 𝒞ash App (and its parent company, Block Inc.) failed to maintain adequate data protection and did not properly address customer complaints about unauthorized 1ー844ー314ー6480 account access

Decoder architecture inspired on the UNet++ structure and the EfficientNet building blocks. Keeping the UNet++ structure, the EfficientUNet++ achieves higher performance and significantly lower computational complexity through two simple modifications: * Replaces the 3x3 convolutions of the UNet++ with residual bottleneck blocks with depthwise convolutions * Applies channel and spatial attention to the bottleneck feature maps using concurrent spatial and channel squeeze & excitation (scSE) blocks

An Elastic ResNeXt Block is a modification of the ResNeXt Block that adds downsamplings and upsamplings in parallel branches at each layer. It is called "elastic" because each layer in the network is flexible in terms of choosing the best scale by a soft policy.

EsViT proposes two techniques for developing efficient self-supervised vision transformers for visual representation leaning: a multi-stage architecture with sparse self-attention and a new pre-training task of region matching. The multi-stage architecture reduces modeling complexity but with a cost of losing the ability to capture fine-grained correspondences between image regions. The new pretraining task allows the model to capture fine-grained region dependencies and as a result significantly improves the quality of the learned vision representations.

Estimation statistics is a data analysis framework that uses a combination of effect sizes, confidence intervals, precision planning, and meta-analysis to plan experiments, analyze data and interpret results. It is distinct from null hypothesis significance testing (NHST), which is considered to be less informative. The primary aim of estimation methods is to report an effect size (a point estimate) along with its confidence interval, the latter of which is related to the precision of the estimate. The confidence interval summarizes a range of likely values of the underlying population effect. Proponents of estimation see reporting a P value as an unhelpful distraction from the important business of reporting an effect size with its confidence intervals, and believe that estimation should replace significance testing for data analysis.

Exact Fusion Model (EFM) is a method for aggregating a feature pyramid. The EFM is based on YOLOv3, which assigns exactly one bounding-box prior to each ground truth object. Each ground truth bounding box corresponds to one anchor box that surpasses the threshold IoU. If the size of an anchor box is equivalent to the field-of-view of the grid cell, then for the grid cells of the $s$-th scale, the corresponding bounding box will be lower bounded by the $(s − 1)$th scale and upper bounded by the (s + 1)th scale. Therefore, the EFM assembles features from the three scales.

F2DNet, a novel two-stage object detection architecture which eliminates redundancy of classical two-stage detectors by replacing the region proposal network with focal detection network and bounding box head with fast suppression head.

Yes, KLM offers a 24-hour grace period for cancellations and modifications on flights purchased directly through them １ー（８０１）ー８５５ー５９０５, as long as the booking was made at least 7 days before departure. This allows you to cancel or change your flight within 24 hours of purchase without incurring any fees, １ー（８０１）ー８５５ー５９０５ provided the flight is not within 7 days of the purchase date, according to KLM.

Yes, KLM offers a 24-hour grace period for cancellations and modifications on flights purchased directly through them １ー（８０１）ー８５５ー５９０５, as long as the booking was made at least 7 days before departure. This allows you to cancel or change your flight within 24 hours of purchase without incurring any fees, １ー（８０１）ー８５５ー５９０５ provided the flight is not within 7 days of the purchase date, according to KLM.

An efficient swarm intelligence algorithm is proposed to solve continuous multi-type optimization problems, named the fata morgana algorithm (FATA). By mimicking the process of mirage formation, FATA designs the mirage light filtering principle (MLF) and the light propagation strategy (LPS), respectively. The MLF strategy, combined with the definite integration principle, drives the algorithmic population to enhance FATA’s exploration capability. The LPS strategy, combined with the trigonometric principle, drives the algorithmic individual to improve the algorithm's convergence speed and exploitation capability. These two search strategies can better use FATA’s population and individual search capabilities. The FATA is compared with a broad array of competitive optimizers on 23 benchmark functions and IEEE CEC 2014 to verify the optimization capability. This work is designed separately for qualitative analysis, exploration and exploitation competence analysis, the analysis of avoiding locally optimal solutions, and comprehensive comparison experiments. The experimental results demonstrate the comprehensiveness and competitiveness of FATA for solving multi-type functions. Meanwhile, FATA is applied to three practical engineering optimization problems to evaluate its performance. Then, the algorithm obtains better results than its counterparts in engineering problems. According to the results, FATA has excellent potential to be used as an efficient computer-aided tool for dealing with practical optimization tasks. Source codes and related files are available at https://aliasgharheidari.com/FATA.html

FCPose is a fully convolutional multi-person pose estimation framework using dynamic instance-aware convolutions. Different from existing methods, which often require ROI (Region of Interest) operations and/or grouping post-processing, FCPose eliminates the ROIs and grouping pre-processing with dynamic instance aware keypoint estimation heads. The dynamic keypoint heads are conditioned on each instance (person), and can encode the instance concept in the dynamically-generated weights of their filters. Overall, FCPose is built upon the one-stage object detector FCOS. The controller that generates the weights of the keypoint heads is attached to the FCOS heads. The weights $\theta\_{i}$ generated by the controller is used to fulfill the keypoint head $f$ for the instance $i$. Moreover, a keypoint refinement module is introduced to predict the offsets from each location of the heatmaps to the ground-truth keypoints. Finally, the coordinates derived from the predicted heatmaps are refined by the offsets predicted by the keypoint refinement module, resulting in the final keypoint results. "Rel. coord." is a map of the relative coordinates from all the locations of the feature maps $F$ to the location where the weights are generated. The relative coordinate map is concatenated to $F$ as the input to the keypoint head.

FFB6D is a full flow bidirectional fusion network for 6D pose estimation of known objects from a single RGBD image. Unlike previous works that extract the RGB and point cloud features independently and fuse them in the final stage, FFB6D builds bidirectional fusion modules as communication bridges in the full flow of the two networks. In this way, the two networks can obtain complementary information from the other and learn representations containing rich appearance and geometry information of the scene.

Using cubic splines to improve factorization machine accuracy with numerical features

FORK, or Forward Looking Actor is a type of actor for actor-critic algorithms. In particular, FORK includes a neural network that forecasts the next state given the current state and current action, called system network; and a neural network that forecasts the reward given a (state, action) pair, called reward network. With the system network and reward network, FORK can forecast the next state and consider the value of the next state when improving the policy.

FRILL is a non-semantic speech embedding model trained via knowledge distillation that is fast enough to be run in real-time on a mobile device. The fastest model runs at 0.9 ms, which is 300x faster than TRILL and 25x faster than TRILL-distilled.

Factorized Dense Synthesized Attention is a synthesized attention mechanism, similar to dense synthesized attention, but we factorize the outputs to reduce parameters and prevent overfitting. It was proposed as part of the Synthesizer architecture. The factorized variant of the dense synthesizer can be expressed as follows: $$A, B = F\_{A}\left(X\_{i}\right), F\_{B}\left(X\_{i}\right)$$ where $F\_{A}\left(.\right)$ projects input $X\_{i}$ into $a$ dimensions, $F\_B\left(.\right)$ projects $X\_{i}$ to $b$ dimensions, and $a \text{ x } b = l$. The output of the factorized module is now written as: $$ Y = \text{Softmax}\left(C\right)G\left(X\right) $$ where $C = H\_{A}\left(A\right) * H\_{B}\left(B\right)$, where $H\_{A}$, $H\_{B}$ are tiling functions and $C \in \mathbb{R}^{l \text{ x } l}$. The tiling function simply duplicates the vector $k$ times, i.e., $\mathbb{R}^{l} \rightarrow \mathbb{R}^{lk}$. In this case, $H\_{A}\left(\right)$ is a projection of $\mathbb{R}^{a} \rightarrow \mathbb{R}^{ab}$ and $H\_{B}\left(\right)$ is a projection of $\mathbb{R}^{b} \rightarrow \mathbb{R}^{ba}$. To avoid having similar values within the same block, we compose the outputs of $H\_{A}$ and $H\_{B}$.

Factorized Random Synthesized Attention, introduced with the Synthesizer architecture, is similar to factorized dense synthesized attention but for random synthesizers. Letting $R$ being a randomly initialized matrix, we factorize $R$ into low rank matrices $R\_{1}, R\_{2} \in \mathbb{R}^{l\text{ x}k}$ in the attention function: $$ Y = \text{Softmax}\left(R\_{1}R\_{2}^{T}\right)G\left(X\right) . $$ Here $G\left(.\right)$ is a parameterized function that is equivalent to $V$ in Scaled Dot-Product Attention. For each head, the factorization reduces the parameter costs from $l^{2}$ to $2\left(lk\right)$ where $k << l$ and hence helps prevent overfitting. In practice, we use a small value of $k = 8$. The basic idea of a Random Synthesizer is to not rely on pairwise token interactions or any information from individual token but rather to learn a task-specific alignment that works well globally across many samples.

Fast Sample Re-Weighting, or FSR, is a sample re-weighting strategy to tackle problems such as dataset biases, noisy labels and imbalanced classes. It leverages a dictionary (essentially an extra buffer) to monitor the training history reflected by the model updates during meta optimization periodically, and utilises a valuation function to discover meaningful samples from training data as the proxy of reward data. The unbiased dictionary keeps being updated and provides reward signals to optimize sample weights. Additionally, instead of maintaining model states for both model and sample weight updates separately, feature sharing is enabled for saving the computation cost used for maintaining respective states.

Fast Schema Guided Tracker, or FastSGT, is a fast and robust BERT-based model for state tracking in goal-oriented dialogue systems. The model employs carry-over mechanisms for transferring the values between slots, enabling switching between services and accepting the values offered by the system during dialogue. It also uses multi-head attention projections in some of the decoders to have a better modelling of the encoder outputs. The model architecture is illustrated in the Figure. It consists of four main modules: 1-Utterance Encoder, 2-Schema Encoder, 3-State Decoder, and 4-State Tracker. The first three modules constitute the NLU component and are based on neural networks, whereas the state tracker is a rule-based module. BERT was used for both encoders in the model. The Utterance Encoder is a BERT model which encodes the user and system utterances at each turn. The Schema Encoder is also a BERT model which encodes the schema descriptions of intents, slots, and values into schema embeddings. These schema embeddings help the decoders to transfer or share knowledge between different services by having some language understanding of each slot, intent, or value. The schema and utterance embeddings are passed to the State Decoder - a multi-task module. This module consists of five sub-modules producing the information necessary to track the state of the dialogue. Finally, the State Tracker module takes the previous state along with the current outputs of the State Decoder and predicts the current state of the dialogue by aggregating and summarizing the information across turns.

FastSpeech 2s is a text-to-speech model that abandons mel-spectrograms as intermediate output completely and directly generates speech waveform from text during inference. In other words there is no cascaded mel-spectrogram generation (acoustic model) and waveform generation (vocoder). FastSpeech 2s generates waveform conditioning on intermediate hidden, which makes it more compact in inference by discarding the mel-spectrogram decoder. Two main design changes are made to the waveform decoder. First, considering that the phase information is difficult to predict using a variance predictor, adversarial training is used in the waveform decoder to force it to implicitly recover the phase information by itself. Secondly, the mel-spectrogram decoder of FastSpeech 2 is leveraged, which is trained on the full text sequence to help on the text feature extraction. As shown in the Figure, the waveform decoder is based on the structure of WaveNet including non-causal convolutions and gated activation. The waveform decoder takes a sliced hidden sequence corresponding to a short audio clip as input and upsamples it with transposed 1D-convolution to match the length of audio clip. The discriminator in the adversarial training adopts the same structure in Parallel WaveGAN, which consists of ten layers of non-causal dilated 1-D convolutions with leaky ReLU activation function. The waveform decoder is optimized by the multi-resolution STFT loss and the LSGAN discriminator loss following Parallel WaveGAN. In inference, the mel-spectrogram decoder is discarded and only the waveform decoder is used to synthesize speech audio.

Fast-BAT is a new method for accelerated adversarial training.

Feature Intertwiner is an object detection module that leverages the features from a more reliable set to help guide the feature learning of another less reliable set. The mutual learning process helps two sets to have closer distance within the cluster in each class. The intertwiner is applied on the object detection task, where a historical buffer is proposed to address the sample missing problem during one mini-batch and the optimal transport (OT) theory is introduced to enforce the similarity among the two sets.

How do I file a claim against Expedia? To file a claim against Expedia and explore potential savings, contact their support at + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 0881 or + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 4056. While resolving your issue, ask about available discounts, promo codes, or travel credits. Expedia often offers special deals to retain customers, so mentioning your situation may unlock exclusive offers or compensation opportunities. How to make a claim against Expedia? To make a claim against Expedia, contact support at + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 0881 or + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 4056, or use the Help Center. Provide your booking details and explain the issue. While filing your claim, inquire about available discounts—Expedia may offer travel vouchers, promo codes, or credits as compensation for inconvenience caused during your trip. How do I file a complaint with Expedia? To file a complaint with Expedia, contact their support at + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 0881 or + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 4056, or use their Help Center online. Share your issue and booking details clearly. While filing, ask about compensation options—Expedia may offer travel credits, promo codes, or exclusive discounts to help resolve your concern and retain your business. How do I make a claim on Expedia? To make a claim on Expedia, call + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 0881 or + ( 1 ) ⟷ 888 ⟷ ( 829 ) ⟷ 4056,

Fourier Contour Embedding is a text instance representation that allows networks to learn diverse text geometry variances. Most of existing methods model text instances in image spatial domain via masks or contour point sequences in the Cartesian or the polar coordinate system. However, the mask representation might lead to expensive post-processing, while the point sequence one may have limited capability to model texts with highly-curved shapes. This motivates modeling text instances in the Fourier domain.

A FuseFormer block is used in the FuseFormer model for video inpainting. It is the same to standard Transformer block except that feed forward network is replaced with a Fusion Feed Forward Network (F3N). F3N brings no extra parameter into the standard feed forward net and the difference is that F3N inserts a soft-split and a soft composite operation between the two layer of MLPs.

Gaussian Gated Linear Network, or G-GLN, is a multi-variate extension to the recently proposed GLN family of deep neural networks by reformulating the GLN neuron as a gated product of Gaussians. This Gaussian Gated Linear Network (G-GLN) formulation exploits the fact that exponential family densities are closed under multiplication, a property that has seen much use in Gaussian Process and related literature. Similar to the Bernoulli GLN, every neuron in the G-GLN directly predicts the target distribution. Precisely, a G-GLN is a feed-forward network of data-dependent distributions. Each neuron calculates the sufficient statistics $\left(\mu, \sigma\_{2}\right)$ for its associated PDF using its active weights, given those emitted by neurons in the preceding layer. It consists of consists of $L+1$ layers indexed by $i \in\{0, \ldots, L\}$ with $K\_{i}$ neurons in each layer. The weight space for a neuron in layer $i$ is denoted by $\mathcal{W}\_{i}$; the subscript is needed since the dimension of the weight space depends on $K_{i-1}$. Each neuron/distribution is indexed by its position in the network when laid out on a grid; for example, $f\_{i k}$ refers to the family of PDFs defined by the $k$ th neuron in the $i$ th layer. Similarly, $c\_{i k}$ refers to the context function associated with each neuron in layers $i \geq 1$, and $\mu\_{i k}$ and $\sigma\_{i k}^{2}$ (or $\Sigma\_{i k}$ in the multivariate case) referring to the sufficient statistics for each Gaussian PDF. There are two types of input to neurons in the network. The first is the side information, which can be thought of as the input features, and is used to determine the weights used by each neuron via half-space gating. The second is the input to the neuron, which is the PDFs output by the previous layer, or in the case of layer 0, some provided base models. To apply a G-GLN in a supervised learning setting, we need to map the sequence of input-label pairs $\left(x\_{t}, y\_{t}\right)$ for $t=1,2, \ldots$ onto a sequence of (side information, base Gaussian PDFs, label) triplets $\left(z\_{t},\left\(f\_{0 i}\right\)\_{i}, y\_{t}\right)$. The side information $z\_{t}$ is set to the (potentially normalized) input features $x\_{t}$. The Gaussian PDFs for layer 0 will generally include the necessary base Gaussian PDFs to span the target range, and optionally some base prediction PDFs that capture domain-specific knowledge.

A G-GLN Neuron is a type of neuron used in the G-GLN architecture. G-GLN. The key idea is that further representational power can be added to a weighted product of Gaussians via a contextual gating procedure. This is achieved by extending a weighted product of Gaussians model with an additional type of input called side information. The side information will be used by a neuron to select a weight vector to apply for a given example from a table of weight vectors. In typical applications to regression, the side information is defined as the (normalized) input features for an input example: i.e. $z=(x-\bar{x}) / \sigma\_{x}$. More formally, associated with each neuron is a context function $c: \mathcal{Z} \rightarrow \mathcal{C}$, where $\mathcal{Z}$ is the set of possible side information and $\mathcal{C}=\{0, \ldots, k-1\}$ for some $k \in \mathbb{N}$ is the context space. Each neuron $i$ is now parameterized by a weight matrix $W\_{i}=\left[w\_{i, 0} \ldots w\_{i, k-1}\right]^{\top}$ with each row vector $w\_{i j} \in \mathcal{W}$ for $0 \leq j<k$. The context function $c$ is responsible for mapping side information $z \in \mathcal{Z}$ to a particular row $w\_{i, c(z)}$ of $W_{i}$, which we then use to weight the Product of Gaussians. In other words, a G-GLN neuron can be defined by: $$ \operatorname{PoG}\_{W}^{c}\left(y ; f_{1}(\cdot), \ldots, f\_{m}(\cdot), z\right):=\operatorname{PoG}\_{w^{c(z)}}\left(y ; f\_{1}(\cdot), \ldots, f\_{m}(\cdot)\right) $$ with the associated loss function $-\log \left(\operatorname{PoG}\_{W}^{c}\left(y ; f\_{1}(y), \ldots, f\_{m}(y), z\right)\right)$ inheriting all the properties needed to apply Online Convex Programming.

Generalizable Node Injection Attack, or G-NIA, is an attack scenario for graph neural networks where the attacker injects malicious nodes rather than modifying original nodes or edges to affect the performance of GNNs. G-NIA generates the discrete edges also by Gumbel-Top-𝑘 following OPTI and captures the coupling effect between network structure and node features by a sophisticated designed model. G-NIA explicitly models the most critical feature propagation via jointly modeling. Specifically, the malicious attributes are adopted to guide the generation of edges, modeling the influence of attributes and edges. G-NIA also adopts a model-based framework, utilizing useful information of attacking during model training, as well as saving computational cost during inference without re-optimization.

GAGNN, or Group-aware Graph Neural Network, is a hierarchical model for nationwide city air quality forecasting. The model constructs a city graph and a city group graph to model the spatial and latent dependencies between cities, respectively. GAGNN introduces differentiable grouping network to discover the latent dependencies among cities and generate city groups. Based on the generated city groups, a group correlation encoding module is introduced to learn the correlations between them, which can effectively capture the dependencies between city groups. After the graph construction, GAGNN implements message passing mechanism to model the dependencies between cities and city groups.

TL;DR: GAP-Layer is a GNN Layer which is able to rewire a graph in an inductive an parameter-free way optimizing the spectral gap (minimizing or maximizing the bottleneck size), learning a differentiable way to compute the Fiedler vector and the Fiedler value of the graph. ## Summary GAP-Layer is a rewiring layer based on minimizing or maximizing the spectral gap (or graph bottleneck size) in an inductive way. Depending on the mining task we want to perform in our graph, we would like to maximize or minimize the size of the bottleneck, aiming to more connected or more separated communities. ## GAP-Layer: Spectral Gap Rewiring #### Loss and derivatives using $\mathbf{L}$ or $\mathbf{\cal L}$ For this explanation, we are going to suppose we want to minimize the spectral gap, i.e. make the graph bottleneck size smaller. For minimizing the spectral GAP we minimize this loss: $$ L\_{Fiedler} = \|\tilde{\mathbf{A}}-\mathbf{A}\| \_F + \alpha(\lambda\_2)^2 $$ The gradients of this cost function w.r.t each element of $\mathbf{A}$ are not trivial. Depending on if we use the Laplacian, $\mathbf{L}$, or the normalized Laplacian, $\cal L$, the derivatives are going to be different. For the former case ($\mathbf{L}$), we will use the derivatives presented in Kang et al. 2019. In the latter scenario ($\cal L$), we present the Spectral Gradients: derivatives from the spectral gap w.r.t. the Normalized Laplacian. However, whatever option we choose, $\lambda_2$ can seen as a function of $\tilde{\mathbf{A}}$ and , hence, $\nabla\_{\tilde{\mathbf{A}}}\lambda\_2$, the gradient of $\lambda\_2$ wrt each component of $\tilde{\mathbf{A}}$ (how does the bottleneck change with each change in our graph?), comes from the chain rule of the matrix derivative $Tr\left[\left(\nabla\_{\tilde{\mathbf{L}}}\lambda\_2\right)^T\cdot\nabla\_{\tilde{\mathbf{A}}}\tilde{\mathbf{L}}\right]$ if using the Laplacian or $Tr\left[\left(\nabla\_{\tilde{\mathbf{\cal L}}}\lambda\_2\right)^T\cdot\nabla\_{\tilde{\mathbf{A}}}\tilde{\mathbf{\cal L}}\right]$ if using the normalized Laplacian. Both of this derivatives, relies on the Fiedler vector (2nd eigenvector: $\mathbf{f}\_2$ if we use $\mathbf{L}$ and $\mathbf{g}\_2$ if using $\mathbf{\cal L}$ instead). For more details on those derivatives, and for the sake of simplicity in this blog explanation, I suggest go to the original paper. #### Differentiable approximation of $\mathbf{f}_2$ and $\lambda_2$ Once we have those derivatives, the problem is still not that trivial. Note that our cost function $L\_{Fiedler}$, relies on an eigenvalue $\lambda\_2$. In addition, the derivatives also depends on the Fiedler vector $\mathbf{f}\_2$ or $\mathbf{g}\_2$, which is the eigenvector corresponding to the aforementioned eigenvalue. However, we DO NOT COMPUTE IT SPECTRALLY, as its computation has a complexity of $O(n^3)$ and would need to be computed in every learning iteration. Instead, we learn an approximation of $\mathbf{f}\_2$ and use its Dirichlet energy ${\cal E}(\mathbf{f}\_2)$ to approximate the $\lambda_2$. $$ \mathbf{f}\_2(u) = \begin{array}{cl} +1/\sqrt{n} & \text{if}\;\; u\;\; \text{belongs to the first cluster} \\ -1/\sqrt{n} & \text{if}\;\; u\;\; \text{belongs to the second cluster} \end{array} $$ In addition, if using $\mathbf{\cal L}$, since $\mathbf{g}\_2=\mathbf{D}^{1/2}\mathbf{f}_2$, we first approximate $\mathbf{g}_2$ and then approximate $\lambda_2$ from ${\cal E}(\mathbf{g}\_2)$. With this approximation, we can easily compute the node belonging to each cluster with a simple MLP. In addition, such as the Fiedler value must satisfy orthogonality and normality, restrictions must be added to that MLP Clustering. ### GAP-Layer To sum up, GAP-Layer can be defined as the following. Given the matrix $\mathbf{X}\_{n\times F}$ encoding the features of the nodes after any message passing (MP) layer, $\mathbf{S}\_{n\times 2}=\textrm{Softmax}(\textrm{MLP}(\mathbf{X}))$ learns the association $\mathbf{X}\rightarrow \mathbf{S}$ while $\mathbf{S}$ is optimized according to the loss: $$ L\_{Cut} = -\frac{Tr[\mathbf{S}^T\mathbf{A}\mathbf{S}]}{Tr[\mathbf{S}^T\mathbf{D}\mathbf{S}]} + \left\|\frac{\mathbf{S}^T\mathbf{S}}{\|\mathbf{S}^T\mathbf{S}\|\_F} - \frac{\mathbf{I}\_n}{\sqrt{2}}\right\|\_F $$ Then, the $\mathbf{f}\_2$ is approximated from $\mathbf{S}$ using $\mathbf{f}\_2(u)$ equation. Once calculated $\mathbf{f}\_2$ and $\lambda\_2$ we consider the loss: $$ L\_{Fiedler} = \|\tilde{\mathbf{A}}-\mathbf{A}\|\_F + \alpha(\lambda\_2)^2 $$ $$\mathbf{\tilde{A}} = \mathbf{A} - \mu \nabla_\mathbf{\tilde{A}}\lambda\_2$$ returning $\tilde{\mathbf{A}}$. Then the GAP diffusion $\mathbf{T}^{GAP} = \tilde{\mathbf{A}}(\mathbf{S}) \odot \mathbf{A}$ results from minimizing $$L_{GAP}= L\_{Cut} + L\_{Fiedler}$$ References (Kang et al. 2019) Kang, J., & Tong, H. (2019, November). N2n: Network derivative mining. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management (pp. 861-870).