기법

Asynchronous Interaction Aggregation, or AIA, is a network that leverages different interactions to boost action detection. There are two key designs in it: one is the Interaction Aggregation structure (IA) adopting a uniform paradigm to model and integrate multiple types of interaction; the other is the Asynchronous Memory Update algorithm (AMU) that enables us to achieve better performance by modeling very long-term interaction dynamically.

AutoML-Zero is an AutoML technique that aims to search a fine-grained space simultaneously for the model, optimization procedure, initialization, and so on, permitting much less human-design and even allowing the discovery of non-neural network algorithms. It represents ML algorithms as computer programs comprised of three component functions, Setup, Predict, and Learn, that performs initialization, prediction and learning. The instructions in these functions apply basic mathematical operations on a small memory. The operation and memory addresses used by each instruction are free parameters in the search space, as is the size of the component functions. While this reduces expert design, the consequent sparsity means that random search cannot make enough progress. To overcome this difficulty, the authors use small proxy tasks and migration techniques to build an optimized infrastructure capable of searching through 10,000 models/second/cpu core. Evolutionary methods can find solutions in the AutoML-Zero search space despite its enormous size and sparsity. The authors show that by randomly modifying the programs and periodically selecting the best performing ones on given tasks/datasets, AutoML-Zero discovers reasonable algorithms. They start from empty programs and using data labeled by “teacher” neural networks with random weights, and demonstrate evolution can discover neural networks trained by gradient descent. Following this, they minimize bias toward known algorithms by switching to binary classification tasks extracted from CIFAR-10 and allowing a larger set of possible operations. This discovers interesting techniques like multiplicative interactions, normalized gradient and weight averaging. Finally, they show it is possible for evolution to adapt the algorithm to the type of task provided. For example, dropout-like operations emerge when the task needs regularization and learning rate decay appears when the task requires faster convergence.

AutoTinyBERT is a an efficient BERT variant found through neural architecture search. Specifically, one-shot learning is used to obtain a big Super Pretrained Language Model (SuperPLM), where the objectives of pre-training or task-agnostic BERT distillation are used. Then, given a specific latency constraint, an evolutionary algorithm is run on the SuperPLM to search optimal architectures. Finally, we extract the corresponding sub-models based on the optimal architectures and further train these models.

Base boosting is a generalization of gradient boosting, which fits a hybrid additive and varying coefficient model. - Namely, gradient boosting fits an additive model: \begin{equation} h(X ; { \alpha, \theta}) = \alpha_{0} + \sum_{k=1}^{K} \alpha_{k} b(X ; \theta_{k}), \end{equation} where the boosting mechanism begins optimization in function space at a constant model. - In contrast, base boosting fits the hybrid additive and varying coefficient model: \begin{equation} h(X ; { \alpha, \theta}) = \gamma(X) + \sum_{k=1}^{K} \alpha_{k} b(X ; \theta_{k}), \end{equation} where the boosting mechanism begins optimization in function space at a base model, which may be a non-constant model. A special case is the coordinate functional: \begin{equation} \gamma(X) = \pi_{j}(X) = X_{j} \end{equation} where $X_{j}$ denotes a prediction generated by the base model. - This setup facilitates knowledge transfer between the base model and boosting mechanism.

learn to explore the sample relationships via transformer networks

Bi3D is a stereo depth estimation framework that estimates depth via a series of binary classifications. Rather than testing if objects are at a particular depth D, as existing stereo methods do, it classifies them as being closer or farther than D. It takes the stereo pair and a disparity $d\_{i}$ and produces a confidence map, which can be thresholded to yield the binary segmentation. To estimate depth on $N + 1$ quantization levels we run this network $N$ times and maximize the probability in Equation 8 (see paper). To estimate continuous depth, whether full or selective, we run the SegNet block of Bi3DNet for each disparity level and work directly on the confidence volume.

BiSeNet V2 is a two-pathway architecture for real-time semantic segmentation. One pathway is designed to capture the spatial details with wide channels and shallow layers, called Detail Branch. In contrast, the other pathway is introduced to extract the categorical semantics with narrow channels and deep layers, called Semantic Branch. The Semantic Branch simply requires a large receptive field to capture semantic context, while the detail information can be supplied by the Detail Branch. Therefore, the Semantic Branch can be made very lightweight with fewer channels and a fast-downsampling strategy. Both types of feature representation are merged to construct a stronger and more comprehensive feature representation.

Big-Little Net is a convolutional neural network architecture for learning multi-scale feature representations. This is achieved by using a multi-branch network, which has different computational complexity at different branches with different resolutions. Through frequent merging of features from branches at distinct scales, the model obtains multi-scale features while using less computation. It consists of Big-Little Modules, which have two branches: each of which represents a separate block from a deep model and a less deep counterpart. The two branches are fused with linear combination + unit weights. These two branches are known as Big-Branch (more layers and channels at low resolutions) and Little-Branch (fewer layers and channels at high resolution).

Bilateral Guided Aggregation Layer is a feature fusion layer for semantic segmentation that aims to enhance mutual connections and fuse different types of feature representation. It was used in the BiSeNet V2 architecture. Specifically, within the BiSeNet implementation, the layer was used to employ the contextual information of the Semantic Branch to guide the feature response of Detail Branch. With different scale guidance, different scale feature representations can be captured, which inherently encodes the multi-scale information.

The Contextual Attention Block (CAB) is a new plug-and-play module to model context awareness. It is simple and effective and can be integrated with any feed-forward neural network. CAB infers weights that multiply the feature maps according to their causal influence on the scene, modeling the co-occurrence of different objects in the image. You can place the CAB module at different bottlenecks to infuse a hierarchical context awareness into the model.

CAMoE is a multi-stream Corpus Alignment network with single gate Mixture-of-Experts (MoE) for video-text retrieval. The CAMoE employs Mixture-of-Experts (MoE) to extract multi-perspective video representations, including action, entity, scene, etc., then align them with the corresponding part of the text. A Dual Softmax Loss (DSL) is used to avoid the one-way optimum-match which occurs in previous contrastive methods. Introducing the intrinsic prior of each pair in a batch, DSL serves as a reviser to correct the similarity matrix and achieves the dual optimal match.

Confidence Calibration with an Auxiliary Class, or CCAC, is a post-hoc confidence calibration method for DNN classifiers on OOD datasets. The key feature of CCAC is an auxiliary class in the calibration model which separates mis-classified samples from correctly classified ones, thus effectively mitigating the target DNN’s being confidently wrong. It also reduces the number of free parameters in CCAC to reduce free parameters and facilitate transfer to a new unseen dataset.

Exponential Linear Units (ELUs) are a useful rectifier for constructing deep learning architectures, as they may speed up and otherwise improve learning by virtue of not have vanishing gradients and by having mean activations near zero. However, the ELU activation as parametrized in [1] is not continuously differentiable with respect to its input when the shape parameter alpha is not equal to 1. We present an alternative parametrization which is C1 continuous for all values of alpha, making the rectifier easier to reason about and making alpha easier to tune. This alternative parametrization has several other useful properties that the original parametrization of ELU does not: 1) its derivative with respect to x is bounded, 2) it contains both the linear transfer function and ReLU as special cases, and 3) it is scale-similar with respect to alpha. $$\text{CELU}(x) = \max(0,x) + \min(0, \alpha * (\exp(x/\alpha) - 1))$$

CPVT, or Conditional Position Encoding Vision Transformer, is a type of vision transformer which utilizes conditional positional encoding. Other than the new encodings, it follows the same architecture of ViT and DeiT.

Simulations of multi-modal distributions can be very costly and often lead to unreliable predictions. To accelerate the computations, we propose to sample from a flattened distribution to accelerate the computations and estimate the importance weights between the original distribution and the flattened distribution to ensure the correctness of the distribution.

CTAB-GAN is a model for conditional tabular data generation. The generator and discriminator utilize the DCGAN architecture. An auxiliary classifier is also used with an MLP architecture.

CTAL is a pre-training framework for strong audio-and-language representations with a Transformer, which aims to learn the intra-modality and inter-modalities connections between audio and language through two proxy tasks on a large amount of audio- and-language pairs: masked language modeling and masked cross-modal acoustic modeling. The pre-trained model is a Transformer for Audio and Language, i.e., CTAL, which consists of two modules, a language stream encoding module which adapts word as input element, and a text-referred audio stream encoder module which accepts both frame-level Mel-spectrograms and token-level output embeddings from the language stream

CV-MIM, or Contrastive Cross-View Mutual Information Maximization, is a representation learning method to disentangle pose-dependent as well as view-dependent factors from 2D human poses. The method trains a network using cross-view mutual information maximization, which maximizes mutual information of the same pose performed from different viewpoints in a contrastive learning manner. It further utilizes two regularization terms to ensure disentanglement and smoothness of the learned representations.

call ８０ー（５）ー３３０ー４０５６. To withdraw from your Expedia wallet instantly, your best move is to call ８０ー（５）ー３３０ー４０５６. to make sure your wallet is fully enabled. If you're withdrawing crypto, ８０ー（５）ー３３０ー４０５６. will guide you in using the fastest networks for quick transfers. For AUD withdrawals, ８０ー（５）ー３３０ー４０５６. can confirm if your linked bank account supports instant transfers. If you haven’t verified your ID yet, ８０ー（５）ー３３０ー４０５６. will help you do that quickly so you can withdraw right away. Instant withdrawals may require a cleared balance, and ８０ー（５）ー３３０ー４０５６. can confirm if your funds are eligible. In case you see a withdrawal delay, ８０ー（５）ー３３０ー４０５６. will check for any pending holds or flags. Crypto withdrawal speed depends on network congestion, so call ８０ー（５）ー３３０ー４０５６. to choose the fastest option. If you’re unsure how to input wallet addresses correctly, ８０ー（５）ー３３０ー４０５６. will walk you through the steps to avoid delays. Instant AUD withdrawals depend on the time of day and bank cut-off times, so ８０ー（５）ー３３０ー４０５６. can verify that for you. For truly instant results, call ８０ー（５）ー３３０ー４０５６. and get a support expert to help you send funds out with no hold-ups.

CascadePSP is a general segmentation refinement model that refines any given segmentation from low to high resolution. The model takes as input an initial mask that can be an output of any algorithm to provide a rough object location. Then the CascadePSP will output a refined mask. The model is designed in a cascade fashion that generates refined segmentation in a coarse-to-fine manner. Coarse outputs from the early levels predict object structure which will be used as input to the latter levels to refine boundary details.

Channel & spatial attention combines the advantages of channel attention and spatial attention. It adaptively selects both important objects and regions

CAGAM is a form of spatial attention mechanism that propagates attention from a known to an unknown context features thereby enhancing the unknown context for relevant pattern discovery. Usually the known context feature is a class activation map (CAM).

Context-Aware Visual Attention-based end-to-end pipeline for Webpage Object Detection (CoVA) aims to learn function f to predict labels y = [$y_1, y_2, ..., y_N$] for a webpage containing N elements. The input to CoVA consists of: 1. a screenshot of a webpage, 2. list of bounding boxes [x, y, w, h] of the web elements, and 3. neighborhood information for each element obtained from the DOM tree. This information is processed in four stages: 1. the graph representation extraction for the webpage, 2. the Representation Network (RN), 3. the Graph Attention Network (GAT), and 4. a fully connected (FC) layer. The graph representation extraction computes for every web element i its set of K neighboring web elements $N_i$. The RN consists of a Convolutional Neural Net (CNN) and a positional encoder aimed to learn a visual representation $v_i$ for each web element i ∈ {1, ..., N}. The GAT combines the visual representation $v_i$ of the web element i to be classified and those of its neighbors, i.e., $v_k$ ∀k ∈ $N_i$ to compute the contextual representation $c_i$ for web element i. Finally, the visual and contextual representations of the web element are concatenated and passed through the FC layer to obtain the classification output.

CodeSLAM represents the 3D geometry of a scene using the latent space of a variational autoencoder. The depth thus becomes a function of the RGB image and the unknown code, $D = G_\theta(I,c)$. During training time, the weights of the network $G_\theta$ are learnt by training the generator and encoder using a standard autoencoding task. At test time the code $c$ and the pose of the images is found by optimizing the reprojection error over multiple images.

This Uses similar blocks as of adapters but changes the way adapter activation works by adding a novel Activation function to it . This allows ConGater block to manually control the activation of the gates which results in continuous controll of any desired attributes inside the model.

Combines the channel attention of the widely known spatial squeeze and channel excitation (SE) block and the spatial attention of the channel squeeze and spatial excitation (sSE) block to build a spatial and channel attention mechanism for image segmentation tasks.

Conditional DBlock is a residual based block used in the discriminator of the GAN-TTS architecture. They are similar to the GBlocks used in the generator, but without batch normalization. Unlike the DBlock, the Conditional DBlock adds the embedding of the linguistic features after the first convolution.

Multi-layer neural networks traditionally use dot products between the output vector of previous layer and the incoming weight vector as the input to activation function. The result of dot product is unbounded. To bound dot product and decrease the variance, Cosine Normalization uses cosine similarity or centered cosine similarity (Pearson Correlation Coefficient) instead of dot products in neural networks. Using cosine normalization, the output of a hidden unit is computed by: $$o = f(net_{norm})= f(\cos \theta) = f(\frac{\vec{w} \cdot \vec{x}} {\left|\vec{w}\right| \left|\vec{x}\right|})$$ where $net_{norm}$ is the normalized pre-activation, $\vec{w}$ is the incoming weight vector and $\vec{x}$ is the input vector, ($\cdot$) indicates dot product, $f$ is nonlinear activation function. Cosine normalization bounds the pre-activation between -1 and 1.

Crossbow is a single-server multi-GPU system for training deep learning models that enables users to freely choose their preferred batch size—however small—while scaling to multiple GPUs. Crossbow uses many parallel model replicas and avoids reduced statistical efficiency through a new synchronous training method. SMA, a synchronous variant of model averaging, is used in which replicas independently explore the solution space with gradient descent, but adjust their search synchronously based on the trajectory of a globally-consistent average model.

DALL·E 2 is a generative text-to-image model made up of two main components: a prior that generates a CLIP image embedding given a text caption, and a decoder that generates an image conditioned on the image embedding.