PyTorchKR Papers

최신 논문

총 1,461편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2202.02520

A rigorous stochastic theory for spike pattern formation in recurrent neural networks with arbitrary connection topologies

Cortical networks exhibit synchronized activity which often occurs in spontaneous events in the form of spike avalanches. Since synchronization has been causally linked to central aspects of brain function such as selective signal processing and integration of stimulus information, participating in an avalanche is a form of a transient synchrony which temporarily creates neural assemblies and hence might especially be useful for implementing flexible information processing. For understanding how assembly formation supports neural computation, it is therefore essential to establish a comprehensive theory of how network structure and dynamics interact to generate specific avalanche patterns and sequences. Here we derive exact avalanche distributions for a finite network of recurrently coupled spiking neurons with arbitrary non-negative interaction weights, which is made possible by formally mapping the model dynamics to a linear, random dynamical system on the $N$-torus and by exploiting self-similarities inherent in the phase space. We introduce the notion of relative unique ergodicity and show that this property is guaranteed if the system is driven by a time-invariant Bernoulli process. This approach allows us not only to provide closed-form analytical expressions for avalanche size, but also to determine the detailed set(s) of units firing in an avalanche (i.e., the avalanche assembly). The underlying dependence between network structure and dynamics is made transparent by expressing the distribution of avalanche assemblies in terms of the induced graph Laplacian. We explore analytical consequences of this dependence and provide illustrating examples.

arXiv:2202.02488

A discussion of stochastic dominance and mean-risk optimal portfolio problems based on mean-variance-mixture models

The classical Markowitz mean-variance model uses variance as a risk measure and calculates frontier portfolios in closed form by using standard optimization techniques. For general mean-risk models such closed form optimal portfolios are difficult to obtain. In this note, assuming returns follow the class of normal mean-variance mixture (NMVM) distributions, we obtain closed form expressions for frontier portfolios under mean-risk criteria when risk is modeled by the general class of law invariant convex risk measures. To achieve this goal, we first present a sufficient condition for the stochastic dominance relation on NMVM models and we apply this result to derive closed form solution for frontier portfolios. Our main result in this paper states that when return vectors follow the class of NMVM distributions the associated mean-risk frontier portfolios can be obtained by optimizing a Markowitz mean-variance model with an appropriately adjusted return vector.

arXiv:2202.01210

Deep Layer-wise Networks Have Closed-Form Weights

There is currently a debate within the neuroscience community over the likelihood of the brain performing backpropagation (BP). To better mimic the brain, training a network \textit{one layer at a time} with only a "single forward pass" has been proposed as an alternative to bypass BP; we refer to these networks as "layer-wise" networks. We continue the work on layer-wise networks by answering two outstanding questions. First, $\textit{do they have a closed-form solution?}$ Second, $\textit{how do we know when to stop adding more layers?}$ This work proves that the Kernel Mean Embedding is the closed-form weight that achieves the network global optimum while driving these networks to converge towards a highly desirable kernel for classification; we call it the $\textit{Neural Indicator Kernel}$.

arXiv:2202.01752

Near-Optimal Learning of Extensive-Form Games with Imperfect Information

This paper resolves the open question of designing near-optimal algorithms for learning imperfect-information extensive-form games from bandit feedback. We present the first line of algorithms that require only $\widetilde{\mathcal{O}}((XA+YB)/\varepsilon^2)$ episodes of play to find an $\varepsilon$-approximate Nash equilibrium in two-player zero-sum games, where $X,Y$ are the number of information sets and $A,B$ are the number of actions for the two players. This improves upon the best known sample complexity of $\widetilde{\mathcal{O}}((X^2A+Y^2B)/\varepsilon^2)$ by a factor of $\widetilde{\mathcal{O}}(\max\{X, Y\})$, and matches the information-theoretic lower bound up to logarithmic factors. We achieve this sample complexity by two new algorithms: Balanced Online Mirror Descent, and Balanced Counterfactual Regret Minimization. Both algorithms rely on novel approaches of integrating \emph{balanced exploration policies} into their classical counterparts. We also extend our results to learning Coarse Correlated Equilibria in multi-player general-sum games.

arXiv:2202.00886공식 코드 · Papers with Code

Accurate calibration of multi-perspective cameras from a generalization of the hand-eye constraint

Multi-perspective cameras are quickly gaining importance in many applications such as smart vehicles and virtual or augmented reality. However, a large system size or absence of overlap in neighbouring fields-of-view often complicate their calibration. We present a novel solution which relies on the availability of an external motion capture system. Our core contribution consists of an extension to the hand-eye calibration problem which jointly solves multi-eye-to-base problems in closed form. We furthermore demonstrate its equivalence to the multi-eye-in-hand problem. The practical validity of our approach is supported by our experiments, indicating that the method is highly efficient and accurate, and outperforms existing closed-form alternatives.

arXiv:2202.00498

Analysis of Linear Time-Varying & Periodic Systems

This thesis applies Floquet theory to analyze linear periodic time-varying (LPTV) systems, represented by a system of ordinary differential equations (ODEs) that depend on a time variable t and have a matrix of coefficients with period T>0. The transition matrix of an LPTV system represented by a square periodic-function matrix A(t)=A(t+T) can be expressed as the product of a square periodic function matrix P(t)=P(t+T) and an exponentiated square matrix of the form Rt, where R is a constant matrix (independent of t). Despite the validity of Floquet theory, it is difficult to find an analytical closed form for the matrices P(t) and R when the transition matrix {\Phi}_A (t,t_0 ) is unknown. In essence, it is difficult to find an analytical solution for an LPTV system (i.e., a closed form for its transition matrix). The research results show that for a given family of periodic matrices A(t), we can compare the powers of {\omega} that multiply the harmonics (i.e., {\omega} is part of the coefficients multiplying the cosine [sine] factors in even [odd] representations or of the exponential factors in complex representations) to determine the matrices P(t) and R. In addition, the results lead to relations between LPTV systems at frequency {\omega} and the associated linear time-invariant system, which is defined by having zero frequency ({\omega}=0).

arXiv:2202.00237

Kernelized Multiplicative Weights for 0/1-Polyhedral Games: Bridging the Gap Between Learning in Extensive-Form and Normal-Form Games

While extensive-form games (EFGs) can be converted into normal-form games (NFGs), doing so comes at the cost of an exponential blowup of the strategy space. So, progress on NFGs and EFGs has historically followed separate tracks, with the EFG community often having to catch up with advances (e.g., last-iterate convergence and predictive regret bounds) from the larger NFG community. In this paper we show that the Optimistic Multiplicative Weights Update (OMWU) algorithm -- the premier learning algorithm for NFGs -- can be simulated on the normal-form equivalent of an EFG in linear time per iteration in the game tree size using a kernel trick. The resulting algorithm, Kernelized OMWU (KOMWU), applies more broadly to all convex games whose strategy space is a polytope with 0/1 integral vertices, as long as the kernel can be evaluated efficiently. In the particular case of EFGs, KOMWU closes several standing gaps between NFG and EFG learning, by enabling direct, black-box transfer to EFGs of desirable properties of learning dynamics that were so far known to be achievable only in NFGs. Specifically, KOMWU gives the first algorithm that guarantees at the same time last-iterate convergence, lower dependence on the size of the game tree than all prior algorithms, and $\tilde{\mathcal{O}}(1)$ regret when followed by all players.

arXiv:2201.12056

Outage performance analysis of RIS-assisted UAV wireless systems under disorientation and misalignment

In this paper, we analyze the performance of a reconfigurable intelligent surface (RIS)-assisted unmanned aerial vehicle (UAV) wireless system that is affected by mixture-gamma small-scale fading, stochastic disorientation, and misalignment, as well as transceivers hardware imperfections. First, we statistically characterize the end-to-end channel for both cases, i.e., in the absence as well as in the presence of disorientation and misalignment, by extracting closed-form formulas for the probability density function (PDF) and the cumulative distribution function (CDF). Building on the aforementioned expressions, we extract novel closed-form expressions for the outage probability (OP) in the absence and the presence of disorientation and misalignment as well as hardware imperfections. In addition, high signal-to-noise ratio OP approximations are derived, leading to the extraction of the diversity order. Finally, an OP floor due to disorientation and misalignment is presented.

arXiv:2201.10375

Zero-Shot Long-Form Voice Cloning with Dynamic Convolution Attention

With recent advancements in voice cloning, the performance of speech synthesis for a target speaker has been rendered similar to the human level. However, autoregressive voice cloning systems still suffer from text alignment failures, resulting in an inability to synthesize long sentences. In this work, we propose a variant of attention-based text-to-speech system that can reproduce a target voice from a few seconds of reference speech and generalize to very long utterances as well. The proposed system is based on three independently trained components: a speaker encoder, synthesizer and universal vocoder. Generalization to long utterances is realized using an energy-based attention mechanism known as Dynamic Convolution Attention, in combination with a set of modifications proposed for the synthesizer based on Tacotron 2. Moreover, effective zero-shot speaker adaptation is achieved by conditioning both the synthesizer and vocoder on a speaker encoder that has been pretrained on a large corpus of diverse data. We compare several implementations of voice cloning systems in terms of speech naturalness, speaker similarity, alignment consistency and ability to synthesize long utterances, and conclude that the proposed model can produce intelligible synthetic speech for extremely long utterances, while preserving a high extent of naturalness and similarity for short texts.

arXiv:2201.05193

`Next Generation' Reservoir Computing: an Empirical Data-Driven Expression of Dynamical Equations in Time-Stepping Form

Next generation reservoir computing based on nonlinear vector autoregression (NVAR) is applied to emulate simple dynamical system models and compared to numerical integration schemes such as Euler and the $2^\text{nd}$ order Runge-Kutta. It is shown that the NVAR emulator can be interpreted as a data-driven method used to recover the numerical integration scheme that produced the data. It is also shown that the approach can be extended to produce high-order numerical schemes directly from data. The impacts of the presence of noise and temporal sparsity in the training set is further examined to gauge the potential use of this method for more realistic applications.

arXiv:2201.02939

Prescribed-time Control for Linear Systems in Canonical Form Via Nonlinear Feedback

For systems in canonical form with nonvanishing uncertainties/disturbances, this work presents an approach to full state regulation within prescribed time irrespective of initial conditions. By introducing the smooth hyperbolic-tangent-like function, a nonlinear and time-varying state feedback control scheme is constructed, which is further extended to address output feedback based prescribed-time regulation by invoking the prescribed-time observer, all are applicable over the entire operational time zone. As an alternative to full state regulation within user-assignable time interval, the proposed method analytically bridges the divide between linear and nonlinear feedback based prescribed-time control, and is able to achieve asymptotic stability, exponential stability and prescribed-time stability with a unified control structure.

arXiv:2201.02647

Data-Efficient Information Extraction from Form-Like Documents

Automating information extraction from form-like documents at scale is a pressing need due to its potential impact on automating business workflows across many industries like financial services, insurance, and healthcare. The key challenge is that form-like documents in these business workflows can be laid out in virtually infinitely many ways; hence, a good solution to this problem should generalize to documents with unseen layouts and languages. A solution to this problem requires a holistic understanding of both the textual segments and the visual cues within a document, which is non-trivial. While the natural language processing and computer vision communities are starting to tackle this problem, there has not been much focus on (1) data-efficiency, and (2) ability to generalize across different document types and languages. In this paper, we show that when we have only a small number of labeled documents for training (~50), a straightforward transfer learning approach from a considerably structurally-different larger labeled corpus yields up to a 27 F1 point improvement over simply training on the small corpus in the target domain. We improve on this with a simple multi-domain transfer learning approach, that is currently in production use, and show that this yields up to a further 8 F1 point improvement. We make the case that data efficiency is critical to enable information extraction systems to scale to handle hundreds of different document-types, and learning good representations is critical to accomplishing this.

arXiv:2201.02495

Sign Language Video Retrieval with Free-Form Textual Queries

Systems that can efficiently search collections of sign language videos have been highlighted as a useful application of sign language technology. However, the problem of searching videos beyond individual keywords has received limited attention in the literature. To address this gap, in this work we introduce the task of sign language retrieval with free-form textual queries: given a written query (e.g., a sentence) and a large collection of sign language videos, the objective is to find the signing video in the collection that best matches the written query. We propose to tackle this task by learning cross-modal embeddings on the recently introduced large-scale How2Sign dataset of American Sign Language (ASL). We identify that a key bottleneck in the performance of the system is the quality of the sign video embedding which suffers from a scarcity of labeled training data. We, therefore, propose SPOT-ALIGN, a framework for interleaving iterative rounds of sign spotting and feature alignment to expand the scope and scale of available training data. We validate the effectiveness of SPOT-ALIGN for learning a robust sign video embedding through improvements in both sign recognition and the proposed video retrieval task.

arXiv:2201.02098

Polytope-form games and Index/Degree Theories for Extensive-form games

We present an index theory of equilibria for extensive form games. This requires developing an index theory for games where the strategy sets of players are general polytopes and their payoff functions are multiaffine in the product of these polytopes. Such polytopes arise from identifying (topologically) equivalent mixed strategies of a normal form game.

arXiv:2112.07126

Analyzing population dynamics models via Sumudu transform

This study demonstrates how to construct the solutions of a more general form of population dynamics models via a blend of variational iterative method with Sumudu transform. In this paper, population growth models are formulated in the form of delay differential equations of pantograph type which is a general form for the existing models. Innovative ways are presented for obtaining the solutions of population growth models where other analytic methods fail. Stimulating procedures for finding patterns and regularities in seemingly chaotic processes have been elucidated in this paper. How, when and why the changes in population sizes occur can be deduced through this study.

arXiv:2201.00776

Observability, Dominance, and Induction in Learning Models

Learning models do not in general imply that weakly dominated strategies are irrelevant or justify the related concept of "forward induction," because rational agents may use dominated strategies as experiments to learn how opponents play, and may not have enough data to rule out a strategy that opponents never use. Learning models also do not support the idea that the selected equilibria should only depend on a game's normal form, even though two games with the same normal form present players with the same decision problems given fixed beliefs about how others play. However, playing the extensive form of a game is equivalent to playing the normal form augmented with the appropriate terminal node partitions so that two games are information equivalent, i.e., the players receive the same feedback about others' strategies.

arXiv:2112.15291

A simple method for estimating the Lorenz curve

Given many popular functional forms for the Lorenz curve do not have a closed-form expression for the Gini index and no study has utilized the observed Gini index to estimate parameter(s) associated with the corresponding parametric functional form, a simple method for estimating the Lorenz curve is introduced. It utilizes 3 indicators, namely, the Gini index and the income shares of the bottom and the top in order to calculate the values of parameters associated with the specified functional form which has a closed-form expression for the Gini index. No error minimization technique is required in order to estimate the Lorenz curve. The data on the Gini index and the income shares of 4 countries that have different level of income inequality, economic, sociological, and regional backgrounds from the United Nations University-World Income Inequality Database are used to illustrate how the simple method works. The overall results indicate that the estimated Lorenz curves fit the actual observations practically well. This simple method could be useful in the situation where the availability of data on income distribution is low. However, if more data on income distribution are available, this study shows that the specified functional form could be used to directly estimate the Lorenz curve. Moreover, the estimated values of the Gini index calculated based on the specified functional form are virtually identical to their actual observations.

arXiv:2112.15182

Product Form of Projection-Based Model Reduction and its Application to Multi-Agent Systems

Orthogonal projection-based reduced order models (PROM) are the output of widely-used model reduction methods. In this work, a novel product form is derived for the reduction error system of these reduced models, and it is shown that any such PROM can be obtained from a sequence of 1-dimensional projection reductions. Investigating the error system product form, we then define interface-invariant PROMs, model order reductions with projection-invariant input and output matrices, and it is shown that for such PROMs the error product systems are strictly proper. Furthermore, exploiting this structure, an analytic $\mathcal{H}_{\infty}$ reduction error bound is obtained and an $\mathcal{H}_{\infty}$ bound optimization problem is defined. Interface-invariant reduced models are natural to graph-based model reduction of multi-agent systems where subsets of agents function as the input and output of the system. In the second part of this study, graph contractions are used as a constructive solution approach to the $\mathcal{H}_{\infty}$ bound optimization problem for multi-agent systems. Edge-based contractions are then utilized in a greedy-edge reduction algorithm and are demonstrated for the model reduction of a first-order Laplacian controlled consensus protocol.

arXiv:2112.10598

A Closed-Form Bound on the Asymptotic Linear Convergence of Iterative Methods via Fixed Point Analysis

In many iterative optimization methods, fixed-point theory enables the analysis of the convergence rate via the contraction factor associated with the linear approximation of the fixed-point operator. While this factor characterizes the asymptotic linear rate of convergence, it does not explain the non-linear behavior of these algorithms in the non-asymptotic regime. In this letter, we take into account the effect of the first-order approximation error and present a closed-form bound on the convergence in terms of the number of iterations required for the distance between the iterate and the limit point to reach an arbitrarily small fraction of the initial distance. Our bound includes two terms: one corresponds to the number of iterations required for the linearized version of the fixed-point operator and the other corresponds to the overhead associated with the approximation error. With a focus on the convergence in the scalar case, the tightness of the proposed bound is proven for positively quadratic first-order difference equations.

arXiv:2112.13432

New Methods & Metrics for LFQA tasks

Long-form question answering (LFQA) tasks require retrieving the documents pertinent to a query, using them to form a paragraph-length answer. Despite considerable progress in LFQA modeling, fundamental issues impede its progress: i) train/validation/test dataset overlap, ii) absence of automatic metrics and iii) generated answers not being "grounded" in retrieved documents. This work addresses every one these critical bottlenecks, contributing natural language inference/generation (NLI/NLG) methods and metrics that make significant strides to their alleviation.