PyTorchKR Papers

최신 논문

총 3,337편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2104.04345

A Graph VAE and Graph Transformer Approach to Generating Molecular Graphs

We propose a combination of a variational autoencoder and a transformer based model which fully utilises graph convolutional and graph pooling layers to operate directly on graphs. The transformer model implements a novel node encoding layer, replacing the position encoding typically used in transformers, to create a transformer with no position information that operates on graphs, encoding adjacent node properties into the edge generation process. The proposed model builds on graph generative work operating on graphs with edge features, creating a model that offers improved scalability with the number of nodes in a graph. In addition, our model is capable of learning a disentangled, interpretable latent space that represents graph properties through a mapping between latent variables and graph properties. In experiments we chose a benchmark task of molecular generation, given the importance of both generated node and edge features. Using the QM9 dataset we demonstrate that our model performs strongly across the task of generating valid, unique and novel molecules. Finally, we demonstrate that the model is interpretable by generating molecules controlled by molecular properties, and we then analyse and visualise the learned latent representation.

arXiv:2104.04329

Learning Position and Target Consistency for Memory-based Video Object Segmentation

This paper studies the problem of semi-supervised video object segmentation(VOS). Multiple works have shown that memory-based approaches can be effective for video object segmentation. They are mostly based on pixel-level matching, both spatially and temporally. The main shortcoming of memory-based approaches is that they do not take into account the sequential order among frames and do not exploit object-level knowledge from the target. To address this limitation, we propose to Learn position and target Consistency framework for Memory-based video object segmentation, termed as LCM. It applies the memory mechanism to retrieve pixels globally, and meanwhile learns position consistency for more reliable segmentation. The learned location response promotes a better discrimination between target and distractors. Besides, LCM introduces an object-level relationship from the target to maintain target consistency, making LCM more robust to error drifting. Experiments show that our LCM achieves state-of-the-art performance on both DAVIS and Youtube-VOS benchmark. And we rank the 1st in the DAVIS 2020 challenge semi-supervised VOS task.

arXiv:2104.03674

Explainability-based Backdoor Attacks Against Graph Neural Networks

Backdoor attacks represent a serious threat to neural network models. A backdoored model will misclassify the trigger-embedded inputs into an attacker-chosen target label while performing normally on other benign inputs. There are already numerous works on backdoor attacks on neural networks, but only a few works consider graph neural networks (GNNs). As such, there is no intensive research on explaining the impact of trigger injecting position on the performance of backdoor attacks on GNNs. To bridge this gap, we conduct an experimental investigation on the performance of backdoor attacks on GNNs. We apply two powerful GNN explainability approaches to select the optimal trigger injecting position to achieve two attacker objectives -- high attack success rate and low clean accuracy drop. Our empirical results on benchmark datasets and state-of-the-art neural network models demonstrate the proposed method's effectiveness in selecting trigger injecting position for backdoor attacks on GNNs. For instance, on the node classification task, the backdoor attack with trigger injecting position selected by GraphLIME reaches over $84 \%$ attack success rate with less than $2.5 \%$ accuracy drop

arXiv:2104.02882

FSR: Accelerating the Inference Process of Transducer-Based Models by Applying Fast-Skip Regularization

Transducer-based models, such as RNN-Transducer and transformer-transducer, have achieved great success in speech recognition. A typical transducer model decodes the output sequence conditioned on the current acoustic state and previously predicted tokens step by step. Statistically, The number of blank tokens in the prediction results accounts for nearly 90\% of all tokens. It takes a lot of computation and time to predict the blank tokens, but only the non-blank tokens will appear in the final output sequence. Therefore, we propose a method named fast-skip regularization, which tries to align the blank position predicted by a transducer with that predicted by a CTC model. During the inference, the transducer model can predict the blank tokens in advance by a simple CTC project layer without many complicated forward calculations of the transducer decoder and then skip them, which will reduce the computation and improve the inference speed greatly. All experiments are conducted on a public Chinese mandarin dataset AISHELL-1. The results show that the fast-skip regularization can indeed help the transducer model learn the blank position alignments. Besides, the inference with fast-skip can be speeded up nearly 4 times with only a little performance degradation.

arXiv:2104.02145

What Will it Take to Fix Benchmarking in Natural Language Understanding?

Evaluation for many natural language understanding (NLU) tasks is broken: Unreliable and biased systems score so highly on standard benchmarks that there is little room for researchers who develop better systems to demonstrate their improvements. The recent trend to abandon IID benchmarks in favor of adversarially-constructed, out-of-distribution test sets ensures that current models will perform poorly, but ultimately only obscures the abilities that we want our benchmarks to measure. In this position paper, we lay out four criteria that we argue NLU benchmarks should meet. We argue most current benchmarks fail at these criteria, and that adversarial data collection does not meaningfully address the causes of these failures. Instead, restoring a healthy evaluation ecosystem will require significant progress in the design of benchmark datasets, the reliability with which they are annotated, their size, and the ways they handle social bias.

arXiv:2104.00380공식 코드 · Papers with Code

Online Multiple Object Tracking with Cross-Task Synergy

Modern online multiple object tracking (MOT) methods usually focus on two directions to improve tracking performance. One is to predict new positions in an incoming frame based on tracking information from previous frames, and the other is to enhance data association by generating more discriminative identity embeddings. Some works combined both directions within one framework but handled them as two individual tasks, thus gaining little mutual benefits. In this paper, we propose a novel unified model with synergy between position prediction and embedding association. The two tasks are linked by temporal-aware target attention and distractor attention, as well as identity-aware memory aggregation model. Specifically, the attention modules can make the prediction focus more on targets and less on distractors, therefore more reliable embeddings can be extracted accordingly for association. On the other hand, such reliable embeddings can boost identity-awareness through memory aggregation, hence strengthen attention modules and suppress drifts. In this way, the synergy between position prediction and embedding association is achieved, which leads to strong robustness to occlusions. Extensive experiments demonstrate the superiority of our proposed model over a wide range of existing methods on MOTChallenge benchmarks. Our code and models are publicly available at https://github.com/songguocode/TADAM.

arXiv:2104.00358

Nine Potential Pitfalls when Designing Human-AI Co-Creative Systems

This position paper examines potential pitfalls on the way towards achieving human-AI co-creation with generative models in a way that is beneficial to the users' interests. In particular, we collected a set of nine potential pitfalls, based on the literature and our own experiences as researchers working at the intersection of HCI and AI. We illustrate each pitfall with examples and suggest ideas for addressing it. Reflecting on all pitfalls, we discuss and conclude with implications for future research directions. With this collection, we hope to contribute to a critical and constructive discussion on the roles of humans and AI in co-creative interactions, with an eye on related assumptions and potential side-effects for creative practices and beyond.

arXiv:2103.17146

Continuous Latent Position Models for Instantaneous Interactions

We create a framework to analyse the timing and frequency of instantaneous interactions between pairs of entities. This type of interaction data is especially common nowadays, and easily available. Examples of instantaneous interactions include email networks, phone call networks and some common types of technological and transportation networks. Our framework relies on a novel extension of the latent position network model: we assume that the entities are embedded in a latent Euclidean space, and that they move along individual trajectories which are continuous over time. These trajectories are used to characterize the timing and frequency of the pairwise interactions. We discuss an inferential framework where we estimate the individual trajectories from the observed interaction data, and propose applications on artificial and real data.

arXiv:2103.16903

UAV Deployment Optimization for Secure Precise Wireless Transmission

This paper develops an unmanned aerial vehicle (UAV) deployment scheme in the context of the directional modulation-based secure precise wireless transmissions (SPWTs), where the optimal UAV position for the SPWT is derived to maximum the secrecy rate (SR) without injecting any artificial noise (AN) signaling. To be specific, the proposed scheme reveals that the optimal position of UAV for maximizing the SR performance has to be placed at the null space of Eves channel, which impels the received energy of the confidential message at the unintended receiver deteriorating to zero whilst benefits the one at the intended receiver achieving its maximum value. Finally, simulation results verify the optimality of our proposed scheme in terms of the achievable SR performance.

arXiv:2103.17191

Modeling Users and Online Communities for Abuse Detection: A Position on Ethics and Explainability

Abuse on the Internet is an important societal problem of our time. Millions of Internet users face harassment, racism, personal attacks, and other types of abuse across various platforms. The psychological effects of abuse on individuals can be profound and lasting. Consequently, over the past few years, there has been a substantial research effort towards automated abusive language detection in the field of NLP. In this position paper, we discuss the role that modeling of users and online communities plays in abuse detection. Specifically, we review and analyze the state of the art methods that leverage user or community information to enhance the understanding and detection of abusive language. We then explore the ethical challenges of incorporating user and community information, laying out considerations to guide future research. Finally, we address the topic of explainability in abusive language detection, proposing properties that an explainable method should aim to exhibit. We describe how user and community information can facilitate the realization of these properties and discuss the effective operationalization of explainability in view of the properties.

arXiv:2103.16409

Deep Hedging of Derivatives Using Reinforcement Learning

This paper shows how reinforcement learning can be used to derive optimal hedging strategies for derivatives when there are transaction costs. The paper illustrates the approach by showing the difference between using delta hedging and optimal hedging for a short position in a call option when the objective is to minimize a function equal to the mean hedging cost plus a constant times the standard deviation of the hedging cost. Two situations are considered. In the first, the asset price follows a geometric Brownian motion. In the second, the asset price follows a stochastic volatility process. The paper extends the basic reinforcement learning approach in a number of ways. First, it uses two different Q-functions so that both the expected value of the cost and the expected value of the square of the cost are tracked for different state/action combinations. This approach increases the range of objective functions that can be used. Second, it uses a learning algorithm that allows for continuous state and action space. Third, it compares the accounting P&L approach (where the hedged position is valued at each step) and the cash flow approach (where cash inflows and outflows are used). We find that a hybrid approach involving the use of an accounting P&L approach that incorporates a relatively simple valuation model works well. The valuation model does not have to correspond to the process assumed for the underlying asset price.

arXiv:1908.00054

Hedging Non-Tradable Risks with Transaction Costs and Price Impact

A risk-averse agent hedges her exposure to a non-tradable risk factor $U$ using a correlated traded asset $S$ and accounts for the impact of her trades on both factors. The effect of the agent's trades on $U$ is referred to as cross-impact. By solving the agent's stochastic control problem, we obtain a closed-form expression for the optimal strategy when the agent holds a linear position in $U$. When the exposure to the non-tradable risk factor $\psi(U_T)$ is non-linear, we provide an approximation to the optimal strategy in closed-form, and prove that the value function is correctly approximated by this strategy when cross-impact and risk-aversion are small. We further prove that when $\psi(U_T)$ is non-linear, the approximate optimal strategy can be written in terms of the optimal strategy for a linear exposure with the size of the position changing dynamically according to the exposure's "Delta" under a particular probability measure.

arXiv:1901.06855

A closed formula for illiquid corporate bonds and an application to the European market

We propose an option approach for pricing bond illiquidity that is reminiscent of the celebrated work of Longstaff (1995) on the non-marketability of some non-dividend-paying shares in IPOs. This approach describes a quite common situation in the fixed income market: it is rather usual to find issuers that, besides liquid benchmark bonds, issue some other bonds that either are placed to a small number of investors in private placements or have a limited issue size. We model interest rate and credit risks via a convenient reduced-form approach. We deduce a simple closed formula for illiquid corporate coupon bond prices when liquid bonds with similar characteristics (e.g. maturity) are present in the market for the same issuer. The key model parameter is the time-to-liquidate a position, i.e. the time that an experienced bond trader takes to liquidate a given position on a corporate coupon bond. We show that illiquid bonds present an additional liquidity spread that depends on the time-to-liquidate aside from bond volatility. We provide a detailed application for two issuers in the European market.

arXiv:2103.09987

Statistical Arbitrage Risk Premium by Machine Learning

How to hedge factor risks without knowing the identities of the factors? We first prove a general theoretical result: even if the exact set of factors cannot be identified, any risky asset can use some portfolio of similar peer assets to hedge against its own factor exposures. A long position of a risky asset and a short position of a "replicate portfolio" of its peers represent that asset's factor residual risk. We coin the expected return of an asset's factor residual risk as its Statistical Arbitrage Risk Premium (SARP). The challenge in empirically estimating SARP is finding the peers for each asset and constructing the replicate portfolios. We use the elastic-net, a machine learning method, to project each stock's past returns onto that of every other stock. The resulting high-dimensional but sparse projection vector serves as investment weights in constructing the stocks' replicate portfolios. We say a stock has high (low) Statistical Arbitrage Risk (SAR) if it has low (high) R-squared with its peers. The key finding is that "unique" stocks have both a higher SARP and higher excess returns than "ubiquitous" stocks: in the cross-section, high SAR stocks have a monthly SARP (monthly excess returns) that is 1.101% (0.710%) greater than low SAR stocks. The average SAR across all stocks is countercyclical. Our results are robust to controlling for various known priced factors and characteristics.

arXiv:2101.01261

Hedging with Bitcoin Futures: The Effect of Liquidation Loss Aversion and Aggressive Trading

We consider the hedging problem where a futures position can be automatically liquidated by the exchange without notice. We derive a semi-closed form for an optimal hedging strategy with dual objectives - to minimise both the variance of the hedged portfolio and the probability of liquidations due to insufficient collateral. The optimal solution depends on the statistical characteristics of the spot and futures extreme returns and parameters that characterise the hedger by loss aversion, choice of leverage and collateral management. An empirical analysis of bitcoin shows that the optimal strategy combines superior hedge effectiveness with a reduction in the probability of liquidation. We compare the performance of seven major direct and inverse hedging instruments traded on five different exchanges, based on minute-level data. We also link this performance to novel speculative trading metrics, which differ markedly between venues.

arXiv:2009.07892

Optimal Order Execution in Intraday Markets: Minimizing Costs in Trade Trajectories

Optimal execution, i.e., the determination of the most cost-effective way to trade volumes in continuous trading sessions, has been a topic of interest in the equity trading world for years. Electricity intraday trading slowly follows this trend but is far from being well-researched. The underlying problem is a very complex one. Energy traders, producers, and electricity wholesale companies receive various position updates from customer businesses, renewable energy production, or plant outages and need to trade these positions in intraday markets. They have a variety of options when it comes to position sizing or timing. Is it better to trade all amounts at once? Should they split orders into smaller pieces? Taking the German continuous hourly intraday market as an example, this paper derives an appropriate model for electricity trading. We present our results from an out-of-sample study and differentiate between simple benchmark models and our more refined optimization approach that takes into account order book depth, time to delivery, and different trading regimes like XBID (Cross-Border Intraday Project) trading. Our paper is highly relevant as it contributes further insight into the academic discussion of algorithmic execution in continuous intraday markets and serves as an orientation for practitioners. Our initial results suggest that optimal execution strategies have a considerable monetary impact.

arXiv:2006.12989

Optimal Hedging in Incomplete Markets

We consider the problem of optimal hedging in an incomplete market with an established pricing kernel. In such a market, prices are uniquely determined, but perfect hedges are usually not available. We work in the rather general setting of a L\'evy-Ito market, where assets are driven jointly by an $n$-dimensional Brownian motion and an independent Poisson random measure on an $n$-dimensional state space. Given a position in need of hedging and the instruments available as hedges, we demonstrate the existence of an optimal hedge portfolio, where optimality is defined by use of an expected least squared-error criterion over a specified time frame, and where the numeraire with respect to which the hedge is optimized is taken to be the benchmark process associated with the designated pricing kernel.

arXiv:2103.02016

Trading Signals In VIX Futures

We propose a new approach for trading VIX futures. We assume that the term structure of VIX futures follows a Markov model. Our trading strategy selects a position in VIX futures by maximizing the expected utility for a day-ahead horizon given the current shape and level of the term structure. Computationally, we model the functional dependence between the VIX futures curve, the VIX futures positions, and the expected utility as a deep neural network with five hidden layers. Out-of-sample backtests of the VIX futures trading strategy suggest that this approach gives rise to reasonable portfolio performance, and to positions in which the investor will be either long or short VIX futures contracts depending on the market environment.

arXiv:2103.00258

Organ-specific Branching Morphogenesis

A common developmental process, called branching morphogenesis, generates the epithelial trees in a variety of organs, including the lungs, kidneys, and glands. How branching morphogenesis can create epithelial architectures of very different shapes and functions remains elusive. In this review, we compare branching morphogenesis and its regulation in lungs and kidneys and discuss the role of signaling pathways, the mesenchyme, the extracellular matrix, and the cytoskeleton as potential organ-specific determinants of branch position, orientation, and shape. Identifying the determinants of branch and organ shape and their adaptation in different organs may reveal how a highly conserved developmental process can be adapted to different structural and functional frameworks and should provide important insights into epithelial morphogenesis and developmental disorders.

arXiv:2101.10080

Selection, recombination, and the ancestral initiation graph

Recently, the selection-recombination equation with a single selected site and an arbitrary number of neutral sites was solved by means of the ancestral selection-recombination graph. Here, we introduce a more accessible approach, namely the ancestral initiation graph. The construction is based on a discretisation of the selection-recombination equation. We apply our method to systematically explain a long-standing observation concerning the dynamics of linkage disequilibrium between two neutral loci hitchhiking along with a selected one. In particular, this clarifies the nontrivial dependence on the position of the selected site.