PyTorchKR Papers

최신 논문

총 716편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2011.12892공식 코드 · Papers with Code

Single point positioning using full and fractional pseudorange measurements from GPS and BDS

In conventional global navigation satellite system (GNSS) receivers, usually full pseudorange measurements are required to complete a single point position fix. However, to obtain full pseudorange measurements takes longer time than for fractional pseudorange measurements. Considering such a fact, in order to shorten the time to first fix and improve the position accuracy during cold or warm start of a dual-constellation GNSS receiver, we propose a positioning algorithm using full and fractional pseudorange measurements from the two navigational constellations. This method uses four full pseudorange measurements from one constellation along with fractional ones from either or both constellations to obtain a potentially rapid position result with an identical accuracy to that of the conventional positioning method using full measurements. Tests with simulated and real Global Positioning System (GPS) and BeiDou Navigation Satellite System (BDS) data demonstrate that the proposed method can generate correct single point position solutions and the position error is identical with the result from the conventional approach using the full pseudorange measurements.

arXiv:2101.11268공식 코드 · Papers with Code

Enquire One's Parent and Child Before Decision: Fully Exploit Hierarchical Structure for Self-Supervised Taxonomy Expansion

Taxonomy is a hierarchically structured knowledge graph that plays a crucial role in machine intelligence. The taxonomy expansion task aims to find a position for a new term in an existing taxonomy to capture the emerging knowledge in the world and keep the taxonomy dynamically updated. Previous taxonomy expansion solutions neglect valuable information brought by the hierarchical structure and evaluate the correctness of merely an added edge, which downgrade the problem to node-pair scoring or mini-path classification. In this paper, we propose the Hierarchy Expansion Framework (HEF), which fully exploits the hierarchical structure's properties to maximize the coherence of expanded taxonomy. HEF makes use of taxonomy's hierarchical structure in multiple aspects: i) HEF utilizes subtrees containing most relevant nodes as self-supervision data for a complete comparison of parental and sibling relations; ii) HEF adopts a coherence modeling module to evaluate the coherence of a taxonomy's subtree by integrating hypernymy relation detection and several tree-exclusive features; iii) HEF introduces the Fitting Score for position selection, which explicitly evaluates both path and level selections and takes full advantage of parental relations to interchange information for disambiguation and self-correction. Extensive experiments show that by better exploiting the hierarchical structure and optimizing taxonomy's coherence, HEF vastly surpasses the prior state-of-the-art on three benchmark datasets by an average improvement of 46.7% in accuracy and 32.3% in mean reciprocal rank.

arXiv:2101.07462공식 코드 · Papers with Code

Deep Reinforcement Learning for Producing Furniture Layout in Indoor Scenes

In the industrial interior design process, professional designers plan the size and position of furniture in a room to achieve a satisfactory design for selling. In this paper, we explore the interior scene design task as a Markov decision process (MDP), which is solved by deep reinforcement learning. The goal is to produce an accurate position and size of the furniture simultaneously for the indoor layout task. In particular, we first formulate the furniture layout task as a MDP problem by defining the state, action, and reward function. We then design the simulated environment and train reinforcement learning agents to produce the optimal layout for the MDP formulation. We conduct our experiments on a large-scale real-world interior layout dataset that contains industrial designs from professional designers. Our numerical results demonstrate that the proposed model yields higher-quality layouts as compared with the state-of-art model. The developed simulator and codes are available at \url{https://github.com/CODE-SUBMIT/simulator1}.

arXiv:2101.06980공식 코드 · Papers with Code

Mitigating the Position Bias of Transformer Models in Passage Re-Ranking

Supervised machine learning models and their evaluation strongly depends on the quality of the underlying dataset. When we search for a relevant piece of information it may appear anywhere in a given passage. However, we observe a bias in the position of the correct answer in the text in two popular Question Answering datasets used for passage re-ranking. The excessive favoring of earlier positions inside passages is an unwanted artefact. This leads to three common Transformer-based re-ranking models to ignore relevant parts in unseen passages. More concerningly, as the evaluation set is taken from the same biased distribution, the models overfitting to that bias overestimate their true effectiveness. In this work we analyze position bias on datasets, the contextualized representations, and their effect on retrieval results. We propose a debiasing method for retrieval datasets. Our results show that a model trained on a position-biased dataset exhibits a significant decrease in re-ranking effectiveness when evaluated on a debiased dataset. We demonstrate that by mitigating the position bias, Transformer-based re-ranking models are equally effective on a biased and debiased dataset, as well as more effective in a transfer-learning setting between two differently biased datasets.

arXiv:2101.02359공식 코드 · Papers with Code

Exploring Text-transformers in AAAI 2021 Shared Task: COVID-19 Fake News Detection in English

In this paper, we describe our system for the AAAI 2021 shared task of COVID-19 Fake News Detection in English, where we achieved the 3rd position with the weighted F1 score of 0.9859 on the test set. Specifically, we proposed an ensemble method of different pre-trained language models such as BERT, Roberta, Ernie, etc. with various training strategies including warm-up,learning rate schedule and k-fold cross-validation. We also conduct an extensive analysis of the samples that are not correctly classified. The code is available at:https://github.com/archersama/3rd-solution-COVID19-Fake-News-Detection-in-English.

arXiv:2101.01389공식 코드 · Papers with Code

Extinction of bistable populations is affected by the shape of their initial spatial distribution

The question of whether biological populations survive or are eventually driven to extinction has long been examined using mathematical models. In this work we study population survival or extinction using a stochastic, discrete lattice-based random walk model where individuals undergo movement, birth and death events. The discrete model is defined on a two-dimensional hexagonal lattice with periodic boundary conditions. A key feature of the discrete model is that crowding effects are introduced by specifying two different crowding functions that govern how local agent density influences movement events and birth/death events. The continuum limit description of the discrete model is a nonlinear reaction-diffusion equation, and we focus on crowding functions that lead to linear diffusion and a bistable source term that is often associated with the strong Allee effect. Using both the discrete and continuum modelling tools we explore the complicated relationship between the long-term survival or extinction of the population and the initial spatial arrangement of the population. In particular, we study different spatial arrangements of initial distributions: (i) a well-mixed initial distribution where the initial density is independent of position in the domain; (ii) a vertical strip initial distribution where the initial density is independent of vertical position in the domain; and, (iii) several forms of two-dimensional initial distributions where the initial population is distributed in regions with different shapes. Our results indicate that the shape of the initial spatial distribution of the population affects extinction of bistable populations. All software required to solve the discrete and continuum models used in this work are available on GitHub.

arXiv:2101.02047공식 코드 · Papers with Code

Unified Learning Approach for Egocentric Hand Gesture Recognition and Fingertip Detection

Head-mounted device-based human-computer interaction often requires egocentric recognition of hand gestures and fingertips detection. In this paper, a unified approach of egocentric hand gesture recognition and fingertip detection is introduced. The proposed algorithm uses a single convolutional neural network to predict the probabilities of finger class and positions of fingertips in one forward propagation. Instead of directly regressing the positions of fingertips from the fully connected layer, the ensemble of the position of fingertips is regressed from the fully convolutional network. Subsequently, the ensemble average is taken to regress the final position of fingertips. Since the whole pipeline uses a single network, it is significantly fast in computation. Experimental results show that the proposed method outperforms the existing fingertip detection approaches including the Direct Regression and the Heatmap-based framework. The effectiveness of the proposed method is also shown in-the-wild scenario as well as in a use-case of virtual reality.

arXiv:2101.00895공식 코드 · Papers with Code

A Mixed Integer Least-Squares Formulation of the GNSS Snapshot Positioning Problem

This paper presents a formulation of Snapshot Positioning as a mixed-integer least-squares problem. In snapshot positioning one estimates a position from code-phase and possibly Doppler observations of a Global Navigation Satellite Systems (GNSS) without knowing the time of departure (timestamp) of the codes. Solving the problem allows a receiver to determine a fix from short radio-frequency snapshots missing the time-stamp information embedded in the GNSS data stream. This is used to reduced the time to first fix in some receivers, and it is used in certain wildlife trackers. This paper presents two new formulations of the problem and an algorithm that solves the resulting mixed-integer least-squares problems. We also show that the new formulations can produce fixes even with huge initial errors, much larger than permitted in Van Diggelen's widely-cited coarse-time navigation method.

arXiv:2012.15228공식 코드 · Papers with Code

Introducing Orthogonal Constraint in Structural Probes

With the recent success of pre-trained models in NLP, a significant focus was put on interpreting their representations. One of the most prominent approaches is structural probing (Hewitt and Manning, 2019), where a linear projection of word embeddings is performed in order to approximate the topology of dependency structures. In this work, we introduce a new type of structural probing, where the linear projection is decomposed into 1. isomorphic space rotation; 2. linear scaling that identifies and scales the most relevant dimensions. In addition to syntactic dependency, we evaluate our method on novel tasks (lexical hypernymy and position in a sentence). We jointly train the probes for multiple tasks and experimentally show that lexical and syntactic information is separated in the representations. Moreover, the orthogonal constraint makes the Structural Probes less vulnerable to memorization.

arXiv:2012.15832공식 코드 · Papers with Code

Shortformer: Better Language Modeling using Shorter Inputs

Increasing the input length has been a driver of progress in language modeling with transformers. We identify conditions where shorter inputs are not harmful, and achieve perplexity and efficiency improvements through two new methods that decrease input length. First, we show that initially training a model on short subsequences before moving on to longer ones both reduces overall training time and, surprisingly, substantially improves perplexity. Second, we show how to improve the efficiency of recurrence methods in transformers, which let models condition on previously processed tokens when generating sequences that exceed the maximal length the transformer can handle at once. Existing methods require computationally expensive relative position embeddings; we introduce a simple alternative of adding absolute position embeddings to queries and keys instead of to word embeddings, which efficiently produces superior results. We show that these recurrent models also benefit from short input lengths. Combining these techniques speeds up training by a factor of 1.65, reduces memory usage, and substantially improves perplexity on WikiText-103, without adding any parameters.

arXiv:2012.13692공식 코드 · Papers with Code

Sparse Adversarial Attack to Object Detection

Adversarial examples have gained tons of attention in recent years. Many adversarial attacks have been proposed to attack image classifiers, but few work shift attention to object detectors. In this paper, we propose Sparse Adversarial Attack (SAA) which enables adversaries to perform effective evasion attack on detectors with bounded \emph{l$_{0}$} norm perturbation. We select the fragile position of the image and designed evasion loss function for the task. Experiment results on YOLOv4 and FasterRCNN reveal the effectiveness of our method. In addition, our SAA shows great transferability across different detectors in the black-box attack setting. Codes are available at \emph{https://github.com/THUrssq/Tianchi04}.

arXiv:2012.09624공식 코드 · Papers with Code

Distance-aware Molecule Graph Attention Network for Drug-Target Binding Affinity Prediction

Accurately predicting the binding affinity between drugs and proteins is an essential step for computational drug discovery. Since graph neural networks (GNNs) have demonstrated remarkable success in various graph-related tasks, GNNs have been considered as a promising tool to improve the binding affinity prediction in recent years. However, most of the existing GNN architectures can only encode the topological graph structure of drugs and proteins without considering the relative spatial information among their atoms. Whereas, different from other graph datasets such as social networks and commonsense knowledge graphs, the relative spatial position and chemical bonds among atoms have significant impacts on the binding affinity. To this end, in this paper, we propose a diStance-aware Molecule graph Attention Network (S-MAN) tailored to drug-target binding affinity prediction. As a dedicated solution, we first propose a position encoding mechanism to integrate the topological structure and spatial position information into the constructed pocket-ligand graph. Moreover, we propose a novel edge-node hierarchical attentive aggregation structure which has edge-level aggregation and node-level aggregation. The hierarchical attentive aggregation can capture spatial dependencies among atoms, as well as fuse the position-enhanced information with the capability of discriminating multiple spatial relations among atoms. Finally, we conduct extensive experiments on two standard datasets to demonstrate the effectiveness of S-MAN.

arXiv:2012.08854공식 코드 · Papers with Code

Using noise resilience for ranking generalization of deep neural networks

Recent papers have shown that sufficiently overparameterized neural networks can perfectly fit even random labels. Thus, it is crucial to understand the underlying reason behind the generalization performance of a network on real-world data. In this work, we propose several measures to predict the generalization error of a network given the training data and its parameters. Using one of these measures, based on noise resilience of the network, we secured 5th position in the predicting generalization in deep learning (PGDL) competition at NeurIPS 2020.

arXiv:2012.07043공식 코드 · Papers with Code

Contrastive Learning of Relative Position Regression for One-Shot Object Localization in 3D Medical Images

Deep learning networks have shown promising performance for accurate object localization in medial images, but require large amount of annotated data for supervised training, which is expensive and expertise burdensome. To address this problem, we present a one-shot framework for organ and landmark localization in volumetric medical images, which does not need any annotation during the training stage and could be employed to locate any landmarks or organs in test images given a support (reference) image during the inference stage. Our main idea comes from that tissues and organs from different human bodies have a similar relative position and context. Therefore, we could predict the relative positions of their non-local patches, thus locate the target organ. Our framework is composed of three parts: (1) A projection network trained to predict the 3D offset between any two patches from the same volume, where human annotations are not required. In the inference stage, it takes one given landmark in a reference image as a support patch and predicts the offset from a random patch to the corresponding landmark in the test (query) volume. (2) A coarse-to-fine framework contains two projection networks, providing more accurate localization of the target. (3) Based on the coarse-to-fine model, we transfer the organ boundingbox (B-box) detection to locating six extreme points along x, y and z directions in the query volume. Experiments on multi-organ localization from head-and-neck (HaN) CT volumes showed that our method acquired competitive performance in real time, which is more accurate and 10^5 times faster than template matching methods with the same setting. Code is available: https://github.com/LWHYC/RPR-Loc.

arXiv:2012.04977공식 코드 · Papers with Code

Hateful Memes Detection via Complementary Visual and Linguistic Networks

Hateful memes are widespread in social media and convey negative information. The main challenge of hateful memes detection is that the expressive meaning can not be well recognized by a single modality. In order to further integrate modal information, we investigate a candidate solution based on complementary visual and linguistic network in Hateful Memes Challenge 2020. In this way, more comprehensive information of the multi-modality could be explored in detail. Both contextual-level and sensitive object-level information are considered in visual and linguistic embedding to formulate the complex multi-modal scenarios. Specifically, a pre-trained classifier and object detector are utilized to obtain the contextual features and region-of-interests (RoIs) from the input, followed by the position representation fusion for visual embedding. While linguistic embedding is composed of three components, i.e., the sentence words embedding, position embedding and the corresponding Spacy embedding (Sembedding), which is a symbol represented by vocabulary extracted by Spacy. Both visual and linguistic embedding are fed into the designed Complementary Visual and Linguistic (CVL) networks to produce the prediction for hateful memes. Experimental results on Hateful Memes Challenge Dataset demonstrate that CVL provides a decent performance, and produces 78:48% and 72:95% on the criteria of AUROC and Accuracy. Code is available at https://github.com/webYFDT/hateful.

arXiv:2012.04350공식 코드 · Papers with Code

MANGO: A Mask Attention Guided One-Stage Scene Text Spotter

Recently end-to-end scene text spotting has become a popular research topic due to its advantages of global optimization and high maintainability in real applications. Most methods attempt to develop various region of interest (RoI) operations to concatenate the detection part and the sequence recognition part into a two-stage text spotting framework. However, in such framework, the recognition part is highly sensitive to the detected results (e.g.), the compactness of text contours). To address this problem, in this paper, we propose a novel Mask AttentioN Guided One-stage text spotting framework named MANGO, in which character sequences can be directly recognized without RoI operation. Concretely, a position-aware mask attention module is developed to generate attention weights on each text instance and its characters. It allows different text instances in an image to be allocated on different feature map channels which are further grouped as a batch of instance features. Finally, a lightweight sequence decoder is applied to generate the character sequences. It is worth noting that MANGO inherently adapts to arbitrary-shaped text spotting and can be trained end-to-end with only coarse position information (e.g.), rectangular bounding box) and text annotations. Experimental results show that the proposed method achieves competitive and even new state-of-the-art performance on both regular and irregular text spotting benchmarks, i.e., ICDAR 2013, ICDAR 2015, Total-Text, and SCUT-CTW1500.

arXiv:2008.12275공식 코드 · Papers with Code

Market-making with reinforcement-learning (SAC)

The paper explores the application of a continuous action space soft actor-critic (SAC) reinforcement learning model to the area of automated market-making. The reinforcement learning agent receives a simulated flow of client trades, thus accruing a position in an asset, and learns to offset this risk by either hedging at simulated "exchange" spreads or by attracting an offsetting client flow by changing offered client spreads (skewing the offered prices). The question of learning minimum spreads that compensate for the risk of taking the position is being investigated. Finally, the agent is posed with a problem of learning to hedge a blended client trade flow resulting from independent price processes (a "portfolio" position). The position penalty method is introduced to improve the convergence. An Open-AI gym-compatible hedge environment is introduced and the Open AI SAC baseline RL engine is being used as a learning baseline.

arXiv:1806.00716공식 코드 · Papers with Code

Weber's law based perception and the stability of animal groups

Group living animals form aggregations and flocks that remain cohesive in spite of internal movements of individuals. This is possible because individual group members repeatedly adjust their position and motion in response to the position and motion of other group members. Here we develop a theoretical approach to address the question, what general features -- if any -- underlie the interaction rules that mediate group stability in animals of all species? We do so by considering how the spatial organisation of a group would change in the complete absence of interactions. Without interactions, a group would disperse in a way that can be easily characterised in terms of Fick's diffusion equations. We can hence address the inverse theoretical problem of finding the individual-level interaction responses that are required to counterbalance diffusion and to preserve group stability. We show that an individual-level response to neighbour densities in the form of Weber's law (a 'universal' law describing the functioning of the sensory systems of animals of all species) results in an 'anti-diffusion' term at the group level. On short time scales, this anti-diffusion restores the initial group configuration in a way which is reminiscent of methods for image deblurring in image processing. We also show that any non-homogeneous, spatial density distribution can be preserved over time if individual movement patterns have the form of a Weber's law response. Weber's law describes the fundamental functioning of perceptual systems. Our study indicates that it is also a necessary -- but not sufficient -- feature of collective interactions in stable animal groups.

arXiv:2012.01118공식 코드 · Papers with Code

Neural Teleportation

In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks. Neural teleportation "teleports" a network to a new position in the weight space and preserves its function. This phenomenon comes directly from the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties. We shed light on surprising and counter-intuitive consequences neural teleportation has on the loss landscape. In particular, we show that teleportation can be used to explore loss level curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process. Our results can be reproduced with the code available here: https://github.com/vitalab/neuralteleportation

arXiv:2011.12589공식 코드 · Papers with Code

Combining Semantic Guidance and Deep Reinforcement Learning For Generating Human Level Paintings

Generation of stroke-based non-photorealistic imagery, is an important problem in the computer vision community. As an endeavor in this direction, substantial recent research efforts have been focused on teaching machines "how to paint", in a manner similar to a human painter. However, the applicability of previous methods has been limited to datasets with little variation in position, scale and saliency of the foreground object. As a consequence, we find that these methods struggle to cover the granularity and diversity possessed by real world images. To this end, we propose a Semantic Guidance pipeline with 1) a bi-level painting procedure for learning the distinction between foreground and background brush strokes at training time. 2) We also introduce invariance to the position and scale of the foreground object through a neural alignment model, which combines object localization and spatial transformer networks in an end to end manner, to zoom into a particular semantic instance. 3) The distinguishing features of the in-focus object are then amplified by maximizing a novel guided backpropagation based focus reward. The proposed agent does not require any supervision on human stroke-data and successfully handles variations in foreground object attributes, thus, producing much higher quality canvases for the CUB-200 Birds and Stanford Cars-196 datasets. Finally, we demonstrate the further efficacy of our method on complex datasets with multiple foreground object instances by evaluating an extension of our method on the challenging Virtual-KITTI dataset. Source code and models are available at https://github.com/1jsingh/semantic-guidance.