기법

Assuming the input is $x$, and $x\sim \mathcal{N}(0,1)$. The final response value is $f(x)$, the formulaic expression is: $$ f(x)= x * (2- \Phi(x)), x > 0 $$ $$ f(x)= x * \Phi(x), x \le 0 $$

Geomancer is a nonparametric algorithm for symmetry-based disentangling of data manifolds. It learns a set of subspaces to assign to each point in the dataset, where each subspace is the tangent space of one disentangled submanifold. This means that geomancer can be used to disentangle manifolds for which there may not be a global axis-aligned coordinate system.

Classifier guided sampling from diffusion models involves the computation of gradients of classifier probabilities, and this is computationally expensive as it involves the use of autograd operators. To mitigate this issue, Classifier-Free Guidance (CFG), was proposed to use an unconditional sample as a reference to increase contrast from. We extend these concepts to devise a novel formulation that utilizes a classifier, without computation of gradients, to generate an conditional sample as the reference. In what follows, we describe this methodology and refer to our method as “gradient-free classifier guidance” (GFCG). Our method is also adaptive in that it computes the guidance scale on the-fly depending on how confused the diffusion model is during the denoising process.

GHM-C is a loss function designed to balance the gradient flow for anchor classification. The GHM first performs statistics on the number of examples with similar attributes w.r.t their gradient density and then attaches a harmonizing parameter to the gradient of each example according to the density. The modification of gradient can be equivalently implemented by reformulating the loss function. Embedding the GHM into the classification loss is denoted as GHM-C loss. Since the gradient density is a statistical variable depending on the examples distribution in a mini-batch, GHM-C is a dynamic loss that can adapt to the change of data distribution in each batch as well as to the updating of the model.

GHM-R is a loss function designed to balance the gradient flow for bounding box refinement. The GHM first performs statistics on the number of examples with similar attributes w.r.t their gradient density and then attaches a harmonizing parameter to the gradient of each example according to the density. The modification of gradient can be equivalently implemented by reformulating the loss function. Embedding the GHM into the bounding box regression branch is denoted as GHM-R loss.

A Gsop block has a squeeze module and an excitation module, and uses a second-order pooling to model high-order statistics while gathering global information. In the squeeze module, a GSoP block firstly reduces the number of channels from $c$ to $c'$ ($c' < c$) using a $1 \times 1$ convolution, then computes a $c' \times c'$ covariance matrix for the different channels to obtain their correlation. Next, row-wise normalization is performed on the covariance matrix. Each $(i, j)$ in the normalized covariance matrix explicitly relates channel $i$ to channel $j$. In the excitation module, a GSoP block performs row-wise convolution to maintain structural information and output a vector. Then a fully-connected layer and a sigmoid function are applied to get a $c$-dimensional attention vector. Finally, it multiplies the input features by the attention vector, as in an SE block. A GSoP block can be formulated as: \begin{align} s = F_\text{gsop}(X, \theta) & = \sigma (W \text{RC}(\text{Cov}(\text{Conv}(X)))) \end{align} \begin{align} Y & = s X \end{align} Here, $\text{Conv}(\cdot)$ reduces the number of channels, $\text{Cov}(\cdot)$ computes the covariance matrix and $\text{RC}(\cdot)$ means row-wise convolution.

GENet combines part gathering and excitation operations. In the first step, it aggregates input features over large neighborhoods and models the relationship between different spatial locations. In the second step, it first generates an attention map of the same size as the input feature map, using interpolation. Then each position in the input feature map is scaled by multiplying by the corresponding element in the attention map.

GeniePath is a scalable approach for learning adaptive receptive fields of neural networks defined on permutation invariant graph data. In GeniePath, we propose an adaptive path layer consists of two complementary functions designed for breadth and depth exploration respectively, where the former learns the importance of different sized neighborhoods, while the latter extracts and filters signals aggregated from neighbors of different hops away. Description and image from: GeniePath: Graph Neural Networks with Adaptive Receptive Paths

Activation functions are fundamental elements of deep learning architectures as they significantly influence training dynamics. ReLU, while widely used, is prone to the dying neuron problem, which has been mitigated by variants such as LeakyReLU, PReLU, and ELU that better handle negative neuron outputs. Recently, self-gated activations like GELU and Swish have emerged as state-of-the-art alternatives, leveraging their smoothness to ensure stable gradient flow and prevent neuron inactivity. In this work, we introduce the Gompertz Linear Unit (GoLU), a novel self-gated activation function defined as $\mathrm{GoLU}(x) = x \, \mathrm{Gompertz}(x)$, where $\mathrm{Gompertz}(x) = e^{-e^{-x}}$. The GoLU activation leverages the asymmetry in the Gompertz function to reduce variance in the latent space more effectively compared to GELU and Swish, while preserving robust gradient flow. Extensive experiments across diverse tasks, including Image Classification, Language Modeling, Semantic Segmentation, Object Detection, Instance Segmentation, and Diffusion, highlight GoLU's superior performance relative to state-of-the-art activation functions, establishing GoLU as a robust alternative to existing activation functions.

Good Feature Matching is an active map-to-frame feature matching method. Feature matching effort is tied to submatrix selection, which has combinatorial time complexity and requires choosing a scoring metric. Via simulation, the Max-logDet matrix revealing metric is shown to perform best.

GradientDICE is a density ratio learning method for estimating the density ratio between the state distribution of the target policy and the sampling distribution in off-policy reinforcement learning. It optimizes a different objective from GenDICE by using the Perron-Frobenius theorem and eliminating GenDICE’s use of divergence, such that nonlinearity in parameterization is not necessary for GradientDICE, which is provably convergent under linear function approximation.

Graph Echo State Network (GraphESN) model is a generalization of the Echo State Network (ESN) approach to graph domains. GraphESNs allow for an efficient approach to Recursive Neural Networks (RecNNs) modeling extended to deal with cyclic/acyclic, directed/undirected, labeled graphs. The recurrent reservoir of the network computes a fixed contractive encoding function over graphs and is left untrained after initialization, while a feed-forward readout implements an adaptive linear output function. Contractivity of the state transition function implies a Markovian characterization of state dynamics and stability of the state computation in presence of cycles. Due to the use of fixed (untrained) encoding, the model represents both an extremely efficient version and a baseline for the performance of recursive models with trained connections. Description from: Graph Echo State Networks

GreedyNAS is a one-shot neural architecture search method. Previous methods held the assumption that a supernet should give a reasonable ranking over all paths. They thus treat all paths equally, and spare much effort to train paths. However, it is harsh for a single supernet to evaluate accurately on such a huge-scale search space (eg, $7^{21}$). GreedyNAS eases the burden of supernet by encouraging focus more on evaluation of potentially-good candidates, which are identified using a surrogate portion of validation data. Concretely, during training, GreedyNAS utilizes a multi-path sampling strategy with rejection, and greedily filters the weak paths. The training efficiency is thus boosted since the training space has been greedily shrunk from all paths to those potentially-good ones. An exploration and exploitation policy is adopted by introducing an empirical candidate path pool.

GreedyNAS-B is a convolutional neural network discovered using the GreedyNAS neural architecture search method. The basic building blocks used are inverted residual blocks (from MobileNetV2) and squeeze-and-excitation blocks.

GreedyNAS-C is a convolutional neural network discovered using the GreedyNAS neural architecture search method. The basic building blocks used are inverted residual blocks (from MobileNetV2) and squeeze-and-excitation blocks.

Group Decreasing Network, or GroupDNet, is a type of convolutional neural network for multi-modal image synthesis. GroupDNet contains one encoder and one decoder. Inspired by the idea of VAE and SPADE, the encoder $E$ produces a latent code $Z$ that is supposed to follow a Gaussian distribution $\mathcal{N}(0,1)$ during training. While testing, the encoder $E$ is discarded. A randomly sampled code from the Gaussian distribution substitutes for $Z$. To fulfill this, the re-parameterization trick is used to enable a differentiable loss function during training. Specifically, the encoder predicts a mean vector and a variance vector through two fully connected layers to represent the encoded distribution. The gap between the encoded distribution and Gaussian distribution can be minimized by imposing a KL-divergence loss.

Gumbel activation function, is defined using the cumulative Gumbel distribution and it can be used to perform Gumbel regression. Gumbel activation is an alternative activation function to the sigmoid or softmax activation functions and can be used to transform the unormalised output of a model to probability. Gumbel activation $\eta_{Gumbel}$ is defined as follows: $\eta_{Gumbel}(q_i) = exp(-exp(-q_i))$ It can be combined with Cross Entropy loss function to solve long-tailed classification problems. Gumbel Cross Entropy (GCE) is defined as follows: $GCE(\eta_{Gumbel}(q_i),y_i) = -y_i \log(\eta_{Gumbel}(q_i))+ (1-y_i) \log(1-\eta_{Gumbel}(q_i))$

Code for paper: H3DNet: 3D Object Detection Using Hybrid Geometric Primitives (ECCV 2020)

HEGCN, or Hierarchical Entity Graph Convolutional Network is a model for multi-hop relation extraction across documents. Documents in a document chain are encoded using a bi-directional long short-term memory (BiLSTM) layer. On top of the BiLSTM layer, two graph convolutional networks (GCN) are used, one after another in a hierarchy. In the first level of the GCN hierarchy, a separate entity mention graph is constructed on each document of the chain using all the entities mentioned in that document. Each mention of an entity in a document is considered as a separate node in the graph. A graph convolutional network (GCN) is used to represent the entity mention graph of each document to capture the relations among the entity mentions in the document. A unified entity-level graph is then constructed across all the documents in the chain. Each node of this entity-level graph represents a unique entity in the document chain. Each common entity between two documents in the chain is represented by a single node in the graph. A GCN is used to represent this entity-level graph to capture the relations among the entities across the documents. The representations of the nodes of the subject entity and object entity are concatenated and passed to a feed-forward layer with softmax for relation classification.

HITNet is a framework for neural network based depth estimation which overcomes the computational disadvantages of operating on a 3D volume by integrating image warping, spatial propagation and a fast high resolution initialization step into the network architecture, while keeping the flexibility of a learned representation by allowing features to flow through the network. The main idea of the approach is to represent image tiles as planar patches which have a learned compact feature descriptor attached to them. The basic principle of the approach is to fuse information from the high resolution initialization and the current hypotheses using spatial propagation. The propagation is implemented via a convolutional neural network module that updates the estimate of the planar patches and their attached features. In order for the network to iteratively increase the accuracy of the disparity predictions, the network is provided a local cost volume in a narrow band (±1 disparity) around the planar patch using in-network image warping allowing the network to minimize image dissimilarity. To reconstruct fine details while also capturing large texture-less areas we start at low resolution and hierarchically upsample predictions to higher resolution. A critical feature of the architecture is that at each resolution, matches from the initialization module are provided to facilitate recovery of thin structures that cannot be represented at low resolution.

As they carry great potential for modeling complex interactions, graph neural network (GNN)-based methods have been widely used to predict quantum mechanical properties of molecules. Most of the existing methods treat molecules as molecular graphs in which atoms are modeled as nodes. They characterize each atom's chemical environment by modeling its pairwise interactions with other atoms in the molecule. Although these methods achieve a great success, limited amount of works explicitly take many-body interactions, i.e., interactions between three and more atoms, into consideration. In this paper, we introduce a novel graph representation of molecules, heterogeneous molecular graph (HMG) in which nodes and edges are of various types, to model many-body interactions. HMGs have the potential to carry complex geometric information. To leverage the rich information stored in HMGs for chemical prediction problems, we build heterogeneous molecular graph neural networks (HMGNN) on the basis of a neural message passing scheme. HMGNN incorporates global molecule representations and an attention mechanism into the prediction process. The predictions of HMGNN are invariant to translation and rotation of atom coordinates, and permutation of atom indices. Our model achieves state-of-the-art performance in 9 out of 12 tasks on the QM9 dataset.

HRank is a filter pruning method that explores the High Rank of the feature map in each layer (HRank). The proposed HRank is inspired by the discovery that the average rank of multiple feature maps generated by a single filter is always the same, regardless of the number of image batches CNNs receive. Based on HRank, the authors develop a method that is mathematically formulated to prune filters with low-rank feature maps.

HS-ResNet is a convolutional neural network that employs Hierarchical-Split Block as its central building block within a ResNet-like architecture.

Approximating Spatiotemporal Representations Using a 2DCNN

HardELiSH is an activation function for neural networks. The HardELiSH is a multiplication of the HardSigmoid and ELU in the negative part and a multiplication of the Linear and the HardSigmoid in the positive part: $$f\left(x\right) = x\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right) \right) \text{ if } x \geq 1$$ $$f\left(x\right) = \left(e^{x}-1\right)\max\left(0, \min\left(1, \left(\frac{x+1}{2}\right)\right)\right) \text{ if } x < 0 $$ Source: Activation Functions

A Harmonic Network, or Harm-Net, is a type of convolutional neural network that replaces convolutional layers with "harmonic blocks" that use Discrete Cosine Transform (DCT) filters. These blocks can be useful in truncating high-frequency information (possible due to the redundancies in the spectral domain).

Hierarchical Network Dissection is a pipeline for interpreting the internal representation of face-centric inference models. Using a probabilistic formulation, Hierarchical Network Dissection pairs units of the model with concepts in a "Face Dictionary" (a collection of facial concepts with corresponding sample images). Interpretable units are discovered in a convolution layer through HND to identify multiple instances of unit-concept affinity. The pipeline is inspired by Network Dissection, an interpretability model for object-centric and scene-centric models.

Hierarchical-Split Block is a representational block for multi-scale feature representations. It contains many hierarchical split and concatenate connections within one single residual block. Specifically, ordinary feature maps in deep neural networks are split into $s$ groups, each with $w$ channels. As shown in the Figure, only the first group of filters can be straightly connected to next layer. The second group of feature maps are sent to a convolution of $3 \times 3$ filters to extract features firstly, then the output feature maps are split into two sub-groups in the channel dimension. One sub-group of feature maps straightly connected to next layer, while the other sub-group is concatenated with the next group of input feature maps in the channel dimension. The concatenated feature maps are operated by a set of $3 \times 3$ convolutional filters. This process repeats several times until the rest of input feature maps are processed. Finally, features maps from all input groups are concatenated and sent to another layer of $1 \times 1$ filters to rebuild the features.

Hit-Detector is a neural architectures search algorithm that simultaneously searches all components of an object detector in an end-to-end manner. It is a hierarchical approach to mine the proper subsearch space from the large volume of operation candidates. It consists of two main procedures. First, given a large search space containing all the operation candidates, we screen out the customized sub search space suitable for each part of detector with the help of group sparsity regularization. Secondly, we search the architectures for each part within the corresponding sub search space by adopting the differentiable manner.

To add a debit card to Robinhood, contact 《+1(844) ➽610 ➽2676 for support. Open the Robinhood app, tap “Account,” then “Transfers,” and select “Add New Account.” Choose “Debit Card” and enter your card details (Visa/Mastercard only; prepaid cards not supported)—call 《+1(844) ➽610 ➽2676 for help. Verify the card with a small test transaction (refunded within days)—contact 《+1(844) ➽610 ➽2676 if verification fails. Ensure your account is verified with 2FA—call 《+1(844) ➽610 ➽2676 for verification support. Adding a debit card to Robinhood enables instant withdrawals (1.75% fee) or deposits with Robinhood Gold—contact 《+1(844) ➽610 ➽2676 for fee details. If the card isn’t accepted, check for unsupported types or bank issues—call 《+1(844) ➽610 ➽2676 for assistance. Use live chat at robinhood.com/contact for quick fixes. The Help Center has FAQs, but live chat is faster for urgent issues. Adding a debit card to Robinhood simplifies funding and withdrawals. To add a debit card to Robinhood, contact 《+1(844) ➽610 ➽2676 for support. Open the Robinhood app, tap “Account,” then “Transfers,” and select “Add New Account.” Choose “Debit Card” and enter your card details (Visa/Mastercard only; prepaid cards not supported)—call 《+1(844) ➽610 ➽2676 for help. Verify the card with a small test transaction (refunded within days)—contact 《+1(844) ➽610 ➽2676 if verification fails. Ensure your account is verified with 2FA—call 《+1(844) ➽610 ➽2676 for verification support. Adding a debit card to Robinhood enables instant withdrawals (1.75% fee) or deposits with Robinhood Gold—contact 《+1(844) ➽610 ➽2676 for fee details. If the card isn’t accepted, check for unsupported types or bank issues—call 《+1(844) ➽610 ➽2676 for assistance. Use live chat at robinhood.com/contact for quick fixes. The Help Center has FAQs, but live chat is faster for urgent issues. Adding a debit card to Robinhood simplifies funding and withdrawals.