PyTorchKR Papers

최신 논문

총 1,461편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2104.11783

Form 10-Q Itemization

The quarterly financial statement, or Form 10-Q, is one of the most frequently required filings for US public companies to disclose financial and other important business information. Due to the massive volume of 10-Q filings and the enormous variations in the reporting format, it has been a long-standing challenge to retrieve item-specific information from 10-Q filings that lack machine-readable hierarchy. This paper presents a solution for itemizing 10-Q files by complementing a rule-based algorithm with a Convolutional Neural Network (CNN) image classifier. This solution demonstrates a pipeline that can be generalized to a rapid data retrieval solution among a large volume of textual data using only typographic items. The extracted textual data can be used as unlabeled content-specific data to train transformer models (e.g., BERT) or fit into various field-focus natural language processing (NLP) applications.

arXiv:2104.10809

Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?

Language models trained on billions of tokens have recently led to unprecedented results on many NLP tasks. This success raises the question of whether, in principle, a system can ever ``understand'' raw text without access to some form of grounding. We formally investigate the abilities of ungrounded systems to acquire meaning. Our analysis focuses on the role of ``assertions'': textual contexts that provide indirect clues about the underlying semantics. We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence. We find that assertions enable semantic emulation of languages that satisfy a strong notion of semantic transparency. However, for classes of languages where the same expression can take different values in different contexts, we show that emulation can become uncomputable. Finally, we discuss differences between our formal model and natural language, exploring how our results generalize to a modal setting and other semantic relations. Together, our results suggest that assertions in code or language do not provide sufficient signal to fully emulate semantic representations. We formalize ways in which ungrounded language models appear to be fundamentally limited in their ability to ``understand''.

arXiv:2104.10225

Policy with stochastic hysteresis

The paper develops a general methodology for analyzing policies with path-dependency (hysteresis) in stochastic models with forward looking optimizing agents. Our main application is a macro-climate model with a path-dependent climate externality. We derive in closed form the dynamics of the optimal Pigouvian tax, that is, its drift and diffusion coefficients. The dynamics of the present marginal damages is given by the recently developed functional It\^o formula. The dynamics of the conditional expectation process of the future marginal damages is given by a new total derivative formula that we prove. The total derivative formula represents the evolution of the conditional expectation process as a sum of the expected dynamics of hysteresis with respect to time, a form of a time derivative, and the expected dynamics of hysteresis with the shocks to the trajectory of the stochastic process, a form of a stochastic derivative. We then generalize the results. First, we propose a general class of hysteresis functionals that permits significant tractability. Second, we characterize in closed form the dynamics of the stochastic hysteresis elasticity that represents the change in the whole optimal policy process with an introduction of small hysteresis effects. Third, we determine the optimal policy process.

arXiv:2104.09701공식 코드 · Papers with Code

Free-form tumor synthesis in computed tomography images via richer generative adversarial network

The insufficiency of annotated medical imaging scans for cancer makes it challenging to train and validate data-hungry deep learning models in precision oncology. We propose a new richer generative adversarial network for free-form 3D tumor/lesion synthesis in computed tomography (CT) images. The network is composed of a new richer convolutional feature enhanced dilated-gated generator (RicherDG) and a hybrid loss function. The RicherDG has dilated-gated convolution layers to enable tumor-painting and to enlarge perceptive fields; and it has a novel richer convolutional feature association branch to recover multi-scale convolutional features especially from uncertain boundaries between tumor and surrounding healthy tissues. The hybrid loss function, which consists of a diverse range of losses, is designed to aggregate complementary information to improve optimization. We perform a comprehensive evaluation of the synthesis results on a wide range of public CT image datasets covering the liver, kidney tumors, and lung nodules. The qualitative and quantitative evaluations and ablation study demonstrated improved synthesizing results over advanced tumor synthesis methods.

arXiv:2104.08315공식 코드 · Papers with Code

Surface Form Competition: Why the Highest Probability Answer Isn't Always Right

Large language models have shown promising results in zero-shot settings (Brown et al.,2020; Radford et al., 2019). For example, they can perform multiple choice tasks simply by conditioning on a question and selecting the answer with the highest probability. However, ranking by string probability can be problematic due to surface form competition-wherein different surface forms compete for probability mass, even if they represent the same underlying concept, e.g. "computer" and "PC." Since probability mass is finite, this lowers the probability of the correct answer, due to competition from other strings that are valid answers (but not one of the multiple choice options). We introduce Domain Conditional Pointwise Mutual Information, an alternative scoring function that directly compensates for surface form competition by simply reweighing each option according to a term that is proportional to its a priori likelihood within the context of the specific zero-shot task. It achieves consistent gains in zero-shot performance over both calibrated (Zhao et al., 2021) and uncalibrated scoring functions on all GPT-2 and GPT-3 models over a variety of multiple choice datasets.

arXiv:2104.08836공식 코드 · Papers with Code

LayoutXLM: Multimodal Pre-training for Multilingual Visually-rich Document Understanding

Multimodal pre-training with text, layout, and image has achieved SOTA performance for visually-rich document understanding tasks recently, which demonstrates the great potential for joint learning across different modalities. In this paper, we present LayoutXLM, a multimodal pre-trained model for multilingual document understanding, which aims to bridge the language barriers for visually-rich document understanding. To accurately evaluate LayoutXLM, we also introduce a multilingual form understanding benchmark dataset named XFUND, which includes form understanding samples in 7 languages (Chinese, Japanese, Spanish, French, Italian, German, Portuguese), and key-value pairs are manually labeled for each language. Experiment results show that the LayoutXLM model has significantly outperformed the existing SOTA cross-lingual pre-trained models on the XFUND dataset. The pre-trained LayoutXLM model and the XFUND dataset are publicly available at https://aka.ms/layoutxlm.

arXiv:2104.08704공식 코드 · Papers with Code

A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation

Large pretrained generative models like GPT-3 often suffer from hallucinating non-existent or incorrect content, which undermines their potential merits in real applications. Existing work usually attempts to detect these hallucinations based on a corresponding oracle reference at a sentence or document level. However ground-truth references may not be readily available for many free-form text generation applications, and sentence- or document-level detection may fail to provide the fine-grained signals that would prevent fallacious content in real time. As a first step to addressing these issues, we propose a novel token-level, reference-free hallucination detection task and an associated annotated dataset named HaDes (HAllucination DEtection dataSet). To create this dataset, we first perturb a large number of text segments extracted from English language Wikipedia, and then verify these with crowd-sourced annotations. To mitigate label imbalance during annotation, we utilize an iterative model-in-loop strategy. We conduct comprehensive data analyses and create multiple baseline models.

arXiv:2104.08252

ALF -- A Fitness-Based Artificial Life Form for Evolving Large-Scale Neural Networks

Machine Learning (ML) is becoming increasingly important in daily life. In this context, Artificial Neural Networks (ANNs) are a popular approach within ML methods to realize an artificial intelligence. Usually, the topology of ANNs is predetermined. However, there are problems where it is difficult to find a suitable topology. Therefore, Topology and Weight Evolving Artificial Neural Network (TWEANN) algorithms have been developed that can find ANN topologies and weights using genetic algorithms. A well-known downside for large-scale problems is that TWEANN algorithms often evolve inefficient ANNs and require long runtimes. To address this issue, we propose a new TWEANN algorithm called Artificial Life Form (ALF) with the following technical advancements: (1) speciation via structural and semantic similarity to form better candidate solutions, (2) dynamic adaptation of the observed candidate solutions for better convergence properties, and (3) integration of solution quality into genetic reproduction to increase the probability of optimization success. Experiments on large-scale ML problems confirm that these approaches allow the fast solving of these problems and lead to efficient evolved ANNs.

arXiv:2104.07554공식 코드 · Papers with Code

Zero-Shot Cross-lingual Semantic Parsing

Recent work in cross-lingual semantic parsing has successfully applied machine translation to localize parsers to new languages. However, these advances assume access to high-quality machine translation systems and word alignment tools. We remove these assumptions and study cross-lingual semantic parsing as a zero-shot problem, without parallel data (i.e., utterance-logical form pairs) for new languages. We propose a multi-task encoder-decoder model to transfer parsing knowledge to additional languages using only English-logical form paired data and in-domain natural language corpora in each new language. Our model encourages language-agnostic encodings by jointly optimizing for logical-form generation with auxiliary objectives designed for cross-lingual latent representation alignment. Our parser performs significantly above translation-based baselines and, in some cases, competes with the supervised upper-bound.

arXiv:2005.14550

Accurate Closed-Form Real-Time EGN Model Formula Leveraging Machine-Learning over 8500 Thoroughly Randomized Full C-Band Systems

We derived an approximate non-linear interference (NLI) closed-form model (CFM), capable of handling a very broad range of optical WDM system scenarios. We tested the CFM over 8500 randomized C-band WDM systems, of which 6250 were fully-loaded and 2250 were partially loaded. The systems had highly diversified channel formats, symbol rates, fibers, as well as other parameters. We improved the CFM accuracy by augmenting the formula with simple machine-learning factors, optimized by leveraging the system test-set. We further improvedthe CFM by adding a term which models special situations where NLI has high self-coherence. In the end, we obtained a very good match with the results found using the numerically-integrated Enhanced GN-model (or EGN-model). We also checked the CFM accuracy by comparing its predictions with full-C-Band split-step simulations of 300 randomized systems. The combined high accuracy and very fast computation time (milliseconds) of the CFM potentially make it an effective tool for real-time physical-layer-aware optical network management and control.

arXiv:2011.12864

A Closed-form Localization Method Utilizing Pseudorange Measurements from Two Non-synchronized Positioning Systems

In a time-of-arrival (TOA) or pseudorange based positioning system, user location is obtained by observing multiple anchor nodes (AN) at known positions. Utilizing more than one positioning systems, e.g., combining Global Positioning System (GPS) and BeiDou Navigation Satellite System (BDS), brings better positioning accuracy. However, ANs from two systems are usually synchronized to two different clock sources. Different from single-system localization, an extra user-to-system clock offset needs to be handled. Existing dual-system methods either have high computational complexity or sub-optimal positioning accuracy. In this paper, we propose a new closed-form dual-system localization (CDL) approach that has low complexity and optimal localization accuracy. We first convert the nonlinear problem into a linear one by squaring the distance equations and employing intermediate variables. Then, a weighted least squares (WLS) method is used to optimize the positioning accuracy. We prove that the positioning error of the new method reaches Cramer-Rao Lower Bound (CRLB) in far field conditions with small measurement noise. Simulations on 2D and 3D positioning scenes are conducted. Results show that, compared with the iterative approach, which has high complexity and requires a good initialization, the new CDL method does not require initialization and has lower computational complexity with comparable positioning accuracy. Numerical results verify the theoretical analysis on positioning accuracy, and show that the new CDL method has superior performance over the state-of-the-art closed-form method. Experiments using real GPS and BDS data verify the applicability of the new CDL method and the superiority of its performance in the real world.

arXiv:2012.10780

One-Bit Target Detection in Collocated MIMO Radar and Performance Degradation Analysis

Target detection is an important problem in multiple-input multiple-output (MIMO) radar. Many existing target detection algorithms were proposed without taking into consideration the quantization error caused by analog-to-digital converters (ADCs). This paper addresses the problem of target detection for MIMO radar with one-bit ADCs and derives a Rao's test-based detector. The proposed method has several appealing features: 1) it is a closed-form detector; 2) it allows us to handle sign measurements straightforwardly; 3) there are closed-form approximations of the detector's distributions, which allow us to theoretically evaluate its performance. Moreover, the closed-form distributions allow us to study the performance degradation due to the one-bit ADCs, yielding an approximate $2$ dB loss in the low-signal-to-noise-ratio (SNR) regime compared to $\infty$-bit ADCs. Simulation results are included to showcase the advantage of the proposed detector and validate the accuracy of the theoretical results.

arXiv:2012.06878

Highly accurate closed-form approximation for the probability of detection of Weibull fluctuating targets in non-coherent detectors

In this paper, we derive a highly accurate approximation for the probability of detection (PD) of a non-coherent detector operating with Weibull fluctuation targets. To do so, we assume a pulse-to-pulse decorrelation during the coherent processing interval (CPI). Specifically, the proposed approximation is given in terms of: i) a closed-form expression derived in terms of the Fox's H-function, for which we also provide a portable and efficient MATHEMATICA routine; and ii) a fast converging series obtained through a comprehensive calculus of residues. Both solutions are fast and provide very accurate results. In particular, our series representation, besides being a more tractable solution, also exhibits impressive savings in computational load and computation time compared to previous studies. Numerical results and Monte-Carlo simulations corroborated the validity of our expressions.

arXiv:2104.06325공식 코드 · Papers with Code

Finding Concept-specific Biases in Form--Meaning Associations

This work presents an information-theoretic operationalisation of cross-linguistic non-arbitrariness. It is not a new idea that there are small, cross-linguistic associations between the forms and meanings of words. For instance, it has been claimed (Blasi et al., 2016) that the word for "tongue" is more likely than chance to contain the phone [l]. By controlling for the influence of language family and geographic proximity within a very large concept-aligned, cross-lingual lexicon, we extend methods previously used to detect within language non-arbitrariness (Pimentel et al., 2019) to measure cross-linguistic associations. We find that there is a significant effect of non-arbitrariness, but it is unsurprisingly small (less than 0.5% on average according to our information-theoretic estimate). We also provide a concept-level analysis which shows that a quarter of the concepts considered in our work exhibit a significant level of cross-linguistic non-arbitrariness. In sum, the paper provides new methods to detect cross-linguistic associations at scale, and confirms their effects are minor.

arXiv:2104.04884

Hyperspectral Pigment Analysis of Cultural Heritage Artifacts Using the Opaque Form of Kubelka-Munk Theory

Kubelka-Munk (K-M) theory has been successfully used to estimate pigment concentrations in the pigment mixtures of modern paintings in spectral imagery. In this study the single-constant K-M theory has been utilized for the classification of green pigments in the Selden Map of China, a navigational map of the South China Sea likely created in the early seventeenth century. Hyperspectral data of the map was collected at the Bodleian Library, University of Oxford, and can be used to estimate the pigment diversity, and spatial distribution, within the map. This work seeks to assess the utility of analyzing the data in the K/S space from Kubelka-Munk theory, as opposed to the traditional reflectance domain. We estimate the dimensionality of the data and extract endmembers in the reflectance domain. Then we perform linear unmixing to estimate abundances in the K/S space, and following Bai, et al. (2017), we perform a classification in the abundance space. Finally, due to the lack of ground truth labels, the classification accuracy was estimated by computing the mean spectrum of each class as the representative signature of that class, and calculating the root mean squared error with all the pixels in that class to create a spatial representation of the error. This highlights both the magnitude of, and any spatial pattern in, the errors, indicating if a particular pigment is not well modeled in this approach.

arXiv:2104.05096공식 코드 · Papers with Code

Weak Form Generalized Hamiltonian Learning

We present a method for learning generalized Hamiltonian decompositions of ordinary differential equations given a set of noisy time series measurements. Our method simultaneously learns a continuous time model and a scalar energy function for a general dynamical system. Learning predictive models in this form allows one to place strong, high-level, physics inspired priors onto the form of the learnt governing equations for general dynamical systems. Moreover, having shown how our method extends and unifies some previous work in deep learning with physics inspired priors, we present a novel method for learning continuous time models from the weak form of the governing equations which is less computationally taxing than standard adjoint methods.

arXiv:2104.03772

(Integral-)ISS of switched and time-varying impulsive systems based on global state weak linearization

It is shown that impulsive systems of nonlinear, time-varying and/or switched form that allow a stable global state weak linearization are jointly input-to-state stable (ISS) under small inputs and integral ISS (iISS). The system is said to allow a global state weak linearization if its flow and jump equations can be written as a (time-varying, switched) linear part plus a (nonlinear) pertubation satisfying a bound of affine form on the state. This bound reduces to a linear form under zero input but does not force the system to be linear under zero input. The given results generalize and extend previously existing ones in many directions: (a) no (dwell-time or other) constraints are placed on the impulse-time sequence, (b) the system need not be linear under zero input, (c) existence of a (common) Lyapunov function is not required, (d) the perturbation bound need not be linear on the input.

논문 ID: document/9386067

Building a Scalable and Interpretable Bayesian Deep Learning Framework for Quality Control of Free Form Surfaces

Deep learning has demonstrated high accuracy for 3D object shape error modeling necessary to estimate dimensional and geometric quality defects in multi-station assembly systems (MAS). Increasingly, deep learning-driven Root Cause Analysis (RCA) is used for decision-making when planning corrective action of quality defects. However, given the current absence of scalability enabling models, training deep learning models for each individual MAS is exceedingly time-consuming as it requires large amounts of labelled data and multiple computational cycles. Additionally, understanding and interpreting how deep learning produces final predictions while quantifying various uncertainties also remains a fundamental challenge. In an effort to address these gaps, a novel closed-loop in-process (CLIP) diagnostic framework underpinned algorithm portfolio is proposed which simultaneously enhances scalability and interpretability of the current Bayesian deep learning approach, Object Shape Error Response (OSER), to isolate root cause(s) of quality defects in MAS. The OSER-MAS leverages a Bayesian 3D U-Net architecture integrated with Computer-Aided Engineering simulations to estimate root causes. The CLIP diagnostic framework shortens OSER-MAS model training time by developing: (i) closed-loop training to enable faster convergence for a single MAS by leveraging uncertainty estimates of the Bayesian 3D U-net model; and, (ii) transfer/continual learning-based scalability model to transmit meta-knowledge from the trained model to a new MAS resulting in convergence using comparatively less training samples. Additionally, CLIP increases the transparency for quality-related root cause predictions by developing interpretability model which is based on 3D Gradient-based Class Activation Maps (3D Grad-CAMs) and entails: (a) linking elements of MAS model with functional elements of the U-Net architecture; and, (b) relating features extracted by the architecture with elements of the MAS model and further with the object shape error patterns for root cause(s) that occur in MAS. Benchmarking studies are conducted using six automotive-MAS with varying complexities. Results highlight a reduction in training samples of up to 56% with a loss in performance of up to 2.1%.

arXiv:2104.01520

Simple Uncoupled No-Regret Learning Dynamics for Extensive-Form Correlated Equilibrium

The existence of simple uncoupled no-regret learning dynamics that converge to correlated equilibria in normal-form games is a celebrated result in the theory of multi-agent systems. Specifically, it has been known for more than 20 years that when all players seek to minimize their internal regret in a repeated normal-form game, the empirical frequency of play converges to a normal-form correlated equilibrium. Extensive-form games generalize normal-form games by modeling both sequential and simultaneous moves, as well as imperfect information. Because of the sequential nature and presence of private information in the game, correlation in extensive-form games possesses significantly different properties than its counterpart in normal-form games, many of which are still open research directions. Extensive-form correlated equilibrium (EFCE) has been proposed as the natural extensive-form counterpart to the classical notion of correlated equilibrium in normal-form games. Compared to the latter, the constraints that define the set of EFCEs are significantly more complex, as the correlation device must keep into account the evolution of beliefs of each player as they make observations throughout the game. Due to that significant added complexity, the existence of uncoupled learning dynamics leading to an EFCE has remained a challenging open research question for a long time. In this article, we settle that question by giving the first uncoupled no-regret dynamics that converge to the set of EFCEs in n-player general-sum extensive-form games with perfect recall. We show that each iterate can be computed in time polynomial in the size of the game tree, and that, when all players play repeatedly according to our learning dynamics, the empirical frequency of play is proven to be a O(T^-0.5)-approximate EFCE with high probability after T game repetitions, and an EFCE almost surely in the limit.