PyTorchKR Papers

최신 논문

총 1,461편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2206.00679

Why Did This Model Forecast This Future? Closed-Form Temporal Saliency Towards Causal Explanations of Probabilistic Forecasts

Forecasting tasks surrounding the dynamics of low-level human behavior are of significance to multiple research domains. In such settings, methods for explaining specific forecasts can enable domain experts to gain insights into the predictive relationships between behaviors. In this work, we introduce and address the following question: given a probabilistic forecasting model how can we identify observed windows that the model considers salient when making its forecasts? We build upon a general definition of information-theoretic saliency grounded in human perception and extend it to forecasting settings by leveraging a crucial attribute of the domain: a single observation can result in multiple valid futures. We propose to express the saliency of an observed window in terms of the differential entropy of the resulting predicted future distribution. In contrast to existing methods that either require explicit training of the saliency mechanism or access to the internal states of the forecasting model, we obtain a closed-form solution for the saliency map for commonly used density functions in probabilistic forecasting. We empirically demonstrate how our framework can recover salient observed windows from head pose features for the sample task of speaking-turn forecasting using a synthesized conversation dataset.

arXiv:2206.00117

Disclosure of Investment Advisor and Broker-Dealer Relationships: Impact on Comprehension and Decision Making

Recently enacted regulations aimed to enhance retail investors' understanding about different types of investment accounts. Toward this goal, the Securities and Exchange Commission (SEC) mandated that SEC-registered investment advisors and broker-dealers provide a brief relationship summary (Form CRS) to retail investors. The present study examines the impact of this regulation on investors and considers its market implications. The effects of Form CRS were evaluated based on three outcome variables: perceived helpfulness, comprehension, and decision making. The study also examined whether personal characteristics, such as investment experience, influenced the disclosure's impact on decision making. Results indicated that participants perceived the disclosure as helpful and it significantly enhanced comprehension about the two types of investment accounts. Critically, participants also showed increased preference and choice for broker-dealers after the disclosure. Increased preference for broker-dealers was associated with greater investment experience, greater comprehension gains, and access to more information from a longer disclosure. These findings suggest that Form CRS may promote informed decision making among retail investors while simultaneously increasing the selection of broker-dealer accounts.

arXiv:2206.00454

Towards Context-Aware Neural Performance-Score Synchronisation

Music can be represented in multiple forms, such as in the audio form as a recording of a performance, in the symbolic form as a computer readable score, or in the image form as a scan of the sheet music. Music synchronisation provides a way to navigate among multiple representations of music in a unified manner by generating an accurate mapping between them, lending itself applicable to a myriad of domains like music education, performance analysis, automatic accompaniment and music editing. Traditional synchronisation methods compute alignment using knowledge-driven and stochastic approaches, typically employing handcrafted features. These methods are often unable to generalise well to different instruments, acoustic environments and recording conditions, and normally assume complete structural agreement between the performances and the scores. This PhD furthers the development of performance-score synchronisation research by proposing data-driven, context-aware alignment approaches, on three fronts: Firstly, I replace the handcrafted features by employing a metric learning based approach that is adaptable to different acoustic settings and performs well in data-scarce conditions. Secondly, I address the handling of structural differences between the performances and scores, which is a common limitation of standard alignment methods. Finally, I eschew the reliance on both feature engineering and dynamic programming, and propose a completely data-driven synchronisation method that computes alignments using a neural framework, whilst also being robust to structural differences between the performances and scores.

arXiv:2205.15624

Scalable Distributional Robustness in a Class of Non Convex Optimization with Guarantees

Distributionally robust optimization (DRO) has shown lot of promise in providing robustness in learning as well as sample based optimization problems. We endeavor to provide DRO solutions for a class of sum of fractionals, non-convex optimization which is used for decision making in prominent areas such as facility location and security games. In contrast to previous work, we find it more tractable to optimize the equivalent variance regularized form of DRO rather than the minimax form. We transform the variance regularized form to a mixed-integer second order cone program (MISOCP), which, while guaranteeing near global optimality, does not scale enough to solve problems with real world data-sets. We further propose two abstraction approaches based on clustering and stratified sampling to increase scalability, which we then use for real world data-sets. Importantly, we provide near global optimality guarantees for our approach and show experimentally that our solution quality is better than the locally optimal ones achieved by state-of-the-art gradient-based methods. We experimentally compare our different approaches and baselines, and reveal nuanced properties of a DRO solution.

arXiv:2205.15294

Efficient Phi-Regret Minimization in Extensive-Form Games via Online Mirror Descent

A conceptually appealing approach for learning Extensive-Form Games (EFGs) is to convert them to Normal-Form Games (NFGs). This approach enables us to directly translate state-of-the-art techniques and analyses in NFGs to learning EFGs, but typically suffers from computational intractability due to the exponential blow-up of the game size introduced by the conversion. In this paper, we address this problem in natural and important setups for the \emph{$\Phi$-Hedge} algorithm -- A generic algorithm capable of learning a large class of equilibria for NFGs. We show that $\Phi$-Hedge can be directly used to learn Nash Equilibria (zero-sum settings), Normal-Form Coarse Correlated Equilibria (NFCCE), and Extensive-Form Correlated Equilibria (EFCE) in EFGs. We prove that, in those settings, the \emph{$\Phi$-Hedge} algorithms are equivalent to standard Online Mirror Descent (OMD) algorithms for EFGs with suitable dilated regularizers, and run in polynomial time. This new connection further allows us to design and analyze a new class of OMD algorithms based on modifying its log-partition function. In particular, we design an improved algorithm with balancing techniques that achieves a sharp $\widetilde{\mathcal{O}}(\sqrt{XAT})$ EFCE-regret under bandit-feedback in an EFG with $X$ information sets, $A$ actions, and $T$ episodes. To our best knowledge, this is the first such rate and matches the information-theoretic lower bound.

arXiv:2205.12590

RSTGen: Imbuing Fine-Grained Interpretable Control into Long-FormText Generators

In this paper, we study the task of improving the cohesion and coherence of long-form text generated by language models. To this end, we propose RSTGen, a framework that utilises Rhetorical Structure Theory (RST), a classical language theory, to control the discourse structure, semantics and topics of generated text. Firstly, we demonstrate our model's ability to control structural discourse and semantic features of generated text in open generation evaluation. Then we experiment on the two challenging long-form text tasks of argument generation and story generation. Evaluation using automated metrics and a metric with high correlation to human evaluation, shows that our model performs competitively against existing models, while offering significantly more controls over generated text than alternative methods.

arXiv:2205.11605공식 코드 · Papers with Code

On Measuring Social Biases in Prompt-Based Multi-Task Learning

Large language models trained on a mixture of NLP tasks that are converted into a text-to-text format using prompts, can generalize into novel forms of language and handle novel tasks. A large body of work within prompt engineering attempts to understand the effects of input forms and prompts in achieving superior performance. We consider an alternative measure and inquire whether the way in which an input is encoded affects social biases promoted in outputs. In this paper, we study T0, a large-scale multi-task text-to-text language model trained using prompt-based learning. We consider two different forms of semantically equivalent inputs: question-answer format and premise-hypothesis format. We use an existing bias benchmark for the former BBQ and create the first bias benchmark in natural language inference BBNLI with hand-written hypotheses while also converting each benchmark into the other form. The results on two benchmarks suggest that given two different formulations of essentially the same input, T0 conspicuously acts more biased in question answering form, which is seen during training, compared to premise-hypothesis form which is unlike its training examples. Code and data are released under https://github.com/feyzaakyurek/bbnli.

arXiv:2205.11535

Identifying magnetic antiskyrmions while they form with convolutional neural networks

Chiral magnets have attracted a large amount of research interest in recent years because they support a variety of topological defects, such as skyrmions and bimerons, and allow for their observation and manipulation through several techniques. They also have a wide range of applications in the field of spintronics, particularly in developing new technologies for memory storage devices. However, the vast amount of data generated in these experimental and theoretical studies requires adequate tools, among which machine learning is crucial. We use a Convolutional Neural Network (CNN) to identify the relevant features in the thermodynamical phases of chiral magnets, including (anti-)skyrmions, bimerons, and helical and ferromagnetic states. We use a flexible multi-label classification framework that can correctly classify states in which different features and phases are mixed. We then train the CNN to predict the features of the final state from snapshots of intermediate states of a lattice Monte Carlo simulation. The trained model allows identifying the different phases reliably and early in the formation process. Thus, the CNN can significantly speed up the large-scale simulations for 3D materials that have been the bottleneck for quantitative studies so far. Moreover, this approach can be applied to the identification of mixed states and emerging features in real-world images of chiral magnets.

arXiv:2205.12031공식 코드 · Papers with Code

Efficient Deviation Types and Learning for Hindsight Rationality in Extensive-Form Games: Corrections

Hindsight rationality is an approach to playing general-sum games that prescribes no-regret learning dynamics for individual agents with respect to a set of deviations, and further describes jointly rational behavior among multiple agents with mediated equilibria. To develop hindsight rational learning in sequential decision-making settings, we formalize behavioral deviations as a general class of deviations that respect the structure of extensive-form games. Integrating the idea of time selection into counterfactual regret minimization (CFR), we introduce the extensive-form regret minimization (EFR) algorithm that achieves hindsight rationality for any given set of behavioral deviations with computation that scales closely with the complexity of the set. We identify behavioral deviation subsets, the partial sequence deviation types, that subsume previously studied types and lead to efficient EFR instances in games with moderate lengths. In addition, we present a thorough empirical analysis of EFR instantiated with different deviation types in benchmark games, where we find that stronger types typically induce better performance.

arXiv:2205.11153

Decoupling multivariate functions using a nonparametric filtered tensor decomposition

Multivariate functions emerge naturally in a wide variety of data-driven models. Popular choices are expressions in the form of basis expansions or neural networks. While highly effective, the resulting functions tend to be hard to interpret, in part because of the large number of required parameters. Decoupling techniques aim at providing an alternative representation of the nonlinearity. The so-called decoupled form is often a more efficient parameterisation of the relationship while being highly structured, favouring interpretability. In this work two new algorithms, based on filtered tensor decompositions of first order derivative information are introduced. The method returns nonparametric estimates of smooth decoupled functions. Direct applications are found in, i.a. the fields of nonlinear system identification and machine learning.

arXiv:2205.09278

Modeling Exemplification in Long-form Question Answering via Retrieval

Exemplification is a process by which writers explain or clarify a concept by providing an example. While common in all forms of writing, exemplification is particularly useful in the task of long-form question answering (LFQA), where a complicated answer can be made more understandable through simple examples. In this paper, we provide the first computational study of exemplification in QA, performing a fine-grained annotation of different types of examples (e.g., hypotheticals, anecdotes) in three corpora. We show that not only do state-of-the-art LFQA models struggle to generate relevant examples, but also that standard evaluation metrics such as ROUGE are insufficient to judge exemplification quality. We propose to treat exemplification as a \emph{retrieval} problem in which a partially-written answer is used to query a large set of human-written examples extracted from a corpus. Our approach allows a reliable ranking-type automatic metrics that correlates well with human evaluation. A human evaluation shows that our model's retrieved examples are more relevant than examples generated from a state-of-the-art LFQA model.

arXiv:2205.08015

Similarity Suppresses Cyclicity: Why Similar Competitors Form Hierarchies

Competitive systems can exhibit both hierarchical (transitive) and cyclic (intransitive) structures. Despite theoretical interest in cyclic competition, which offers richer dynamics, and occupies a larger subset of the space of possible competitive systems, most real-world systems are predominantly transitive. Why? Here, we introduce a generic mechanism which promotes transitivity, even when there is ample room for cyclicity. Consider a competitive system where outcomes are mediated by competitor attributes via a performance function. We demonstrate that, if competitive outcomes depend smoothly on competitor attributes, then similar competitors compete transitively. We quantify the rate of convergence to transitivity given the similarity of the competitors and the smoothness of the performance function. Thus, we prove the adage regarding apples and oranges. Similar objects admit well ordered comparisons. Diverse objects may not. To test that theory, we run a series of evolution experiments designed to mimic genetic training algorithms. We consider a series of canonical bimatrix games and an ensemble of random performance functions that demonstrate the generality of our mechanism, even when faced with highly cyclic games. We vary the training parameters controlling the evolution process, and the shape parameters controlling the performance function, to evaluate the robustness of our results. These experiments illustrate that, if competitors evolve to optimize performance, then their traits may converge, leading to transitivity.

arXiv:2205.07223

Sample-Efficient Learning of Correlated Equilibria in Extensive-Form Games

Imperfect-Information Extensive-Form Games (IIEFGs) is a prevalent model for real-world games involving imperfect information and sequential plays. The Extensive-Form Correlated Equilibrium (EFCE) has been proposed as a natural solution concept for multi-player general-sum IIEFGs. However, existing algorithms for finding an EFCE require full feedback from the game, and it remains open how to efficiently learn the EFCE in the more challenging bandit feedback setting where the game can only be learned by observations from repeated playing. This paper presents the first sample-efficient algorithm for learning the EFCE from bandit feedback. We begin by proposing $K$-EFCE -- a more generalized definition that allows players to observe and deviate from the recommended actions for $K$ times. The $K$-EFCE includes the EFCE as a special case at $K=1$, and is an increasingly stricter notion of equilibrium as $K$ increases. We then design an uncoupled no-regret algorithm that finds an $\varepsilon$-approximate $K$-EFCE within $\widetilde{\mathcal{O}}(\max_{i}X_iA_i^{K}/\varepsilon^2)$ iterations in the full feedback setting, where $X_i$ and $A_i$ are the number of information sets and actions for the $i$-th player. Our algorithm works by minimizing a wide-range regret at each information set that takes into account all possible recommendation histories. Finally, we design a sample-based variant of our algorithm that learns an $\varepsilon$-approximate $K$-EFCE within $\widetilde{\mathcal{O}}(\max_{i}X_iA_i^{K+1}/\varepsilon^2)$ episodes of play in the bandit feedback setting. When specialized to $K=1$, this gives the first sample-efficient algorithm for learning EFCE from bandit feedback.

arXiv:2205.06807공식 코드 · Papers with Code

Transformation-Interaction-Rational Representation for Symbolic Regression

Symbolic Regression searches for a function form that approximates a dataset often using Genetic Programming. Since there is usually no restriction to what form the function can have, Genetic Programming may return a hard to understand model due to non-linear function chaining or long expressions. A novel representation called Interaction-Transformation was recently proposed to alleviate this problem. In this representation, the function form is restricted to an affine combination of terms generated as the application of a single univariate function to the interaction of selected variables. This representation obtained competing solutions on standard benchmarks. Despite the initial success, a broader set of benchmarking functions revealed the limitations of the constrained representation. In this paper we propose an extension to this representation, called Transformation-Interaction-Rational representation that defines a new function form as the rational of two Interaction-Transformation functions. Additionally, the target variable can also be transformed with an univariate function. The main goal is to improve the approximation power while still constraining the overall complexity of the expression. We tested this representation with a standard Genetic Programming with crossover and mutation. The results show a great improvement when compared to its predecessor and a state-of-the-art performance for a large benchmark.

arXiv:2205.06000

Accounting for the Sequential Nature of States to Learn Features for Reinforcement Learning

In this work, we investigate the properties of data that cause popular representation learning approaches to fail. In particular, we find that in environments where states do not significantly overlap, variational autoencoders (VAEs) fail to learn useful features. We demonstrate this failure in a simple gridworld domain, and then provide a solution in the form of metric learning. However, metric learning requires supervision in the form of a distance function, which is absent in reinforcement learning. To overcome this, we leverage the sequential nature of states in a replay buffer to approximate a distance metric and provide a weak supervision signal, under the assumption that temporally close states are also semantically similar. We modify a VAE with triplet loss and demonstrate that this approach is able to learn useful features for downstream tasks, without additional supervision, in environments where standard VAEs fail.

arXiv:2205.05489

Hull and White and Alòs type formulas for barrier options in stochastic volatility models with nonzero correlation

Two novel closed-form formulas for the price of barrier options in stochastic volatility models with zero interest rate and dividend yield but nonzero correlation between the asset and its instantaneous volatility are derived. The first is a Hull and White type formula, and the second is a decomposition formula similar in form to the Al\`os decomposition for vanilla options. A model-free approximation is also given.

arXiv:2205.04212

Think outside the search box: A comparative study of visual and form-based query builders

Knowledge workers such as healthcare information professionals, legal researchers, and librarians need to create and execute search strategies that are comprehensive, transparent, and reproducible. The traditional solution is to use proprietary query building tools provided by literature database vendors. In the majority of cases, these query builders are designed using a form-based paradigm that requires the user to enter keywords and ontology terms on a line-by-line basis and then combine them using Boolean operators. However, recent years have witnessed significant changes in human-computer interaction technologies and users can now engage with online information systems using a variety of novel data visualisation techniques. In this paper, we evaluate a new approach to query building in which users express concepts as objects on a visual canvas and compare this with traditional form-based query building in a lab-based user study. The results demonstrate the potential of visual interfaces to mitigate some of the shortcomings associated with form-based interfaces and encourage more exploratory search behaviour. They also demonstrate the value of having a temporary 'scratch' space in query formulation. In addition, the findings highlight an ongoing need for transparency and reproducibility in professional search and raise further questions around how these properties may best be supported.

arXiv:2205.01175

Integrating Structural and Reduced-Form Methods in Empirical Finance

I discuss various ways in which inference based on the estimation of the parameters of statistical models (reduced-form estimation) can be combined with inference based on the estimation of the parameters of economic models (structural estimation). I discuss five basic categories of integration: directly combining the two methods, using statistical models to simplify structural estimation, using structural estimation to extend the validity of reduced-form results, using reduced-form techniques to assess the external validity of structural estimations, and using structural estimation as a sample selection remedy. I illustrate each of these methods with examples from corporate finance, banking, and personal finance.

arXiv:2205.00573

Pricing Path-dependent Options under Stochastic Volatility via Mellin Transform

In this paper, we derive closed-form formulas of first-order approximation for down-and-out barrier and floating strike lookback put option prices under a stochastic volatility model, by using an asymptotic approach. To find the explicit closed-form formulas for the zero-order term and the first-order correction term, we use Mellin transform. We also conduct a sensitivity analysis on these formulas, and compare the option prices calculated by them with those generated by Monte-Carlo simulation.