PyTorchKR Papers

최신 논문

총 238편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:1507.05117

Fast Approximate Bayesian Computation for Estimating Parameters in Differential Equations

Approximate Bayesian computation (ABC) using a sequential Monte Carlo method provides a comprehensive platform for parameter estimation, model selection and sensitivity analysis in differential equations. However, this method, like other Monte Carlo methods, incurs a significant computational cost as it requires explicit numerical integration of differential equations to carry out inference. In this paper we propose a novel method for circumventing the requirement of explicit integration by using derivatives of Gaussian processes to smooth the observations from which parameters are estimated. We evaluate our methods using synthetic data generated from model biological systems described by ordinary and delay differential equations. Upon comparing the performance of our method to existing ABC techniques, we demonstrate that it produces comparably reliable parameter estimates at a significantly reduced execution time.

arXiv:1604.02668

Distance for Functional Data Clustering Based on Smoothing Parameter Commutation

We propose a novel method to determine the dissimilarity between subjects for functional data clustering. Spline smoothing or interpolation is common to deal with data of such type. Instead of estimating the best-representing curve for each subject as fixed during clustering, we measure the dissimilarity between subjects based on varying curve estimates with commutation of smoothing parameters pair-by-pair (of subjects). The intuitions are that smoothing parameters of smoothing splines reflect inverse signal-to-noise ratios and that applying an identical smoothing parameter the smoothed curves for two similar subjects are expected to be close. The effectiveness of our proposal is shown through simulations comparing to other dissimilarity measures. It also has several pragmatic advantages. First, missing values or irregular time points can be handled directly, thanks to the nature of smoothing splines. Second, conventional clustering method based on dissimilarity can be employed straightforward, and the dissimilarity also serves as a useful tool for outlier detection. Third, the implementation is almost handy since subroutines for smoothing splines and numerical integration are widely available. Fourth, the computational complexity does not increase and is parallel with that in calculating Euclidean distance between curves estimated by smoothing splines.

arXiv:1608.02702

Steerable Principal Components for Space-Frequency Localized Images

This paper describes a fast and accurate method for obtaining steerable principal components from a large dataset of images, assuming the images are well localized in space and frequency. The obtained steerable principal components are optimal for expanding the images in the dataset and all of their rotations. The method relies upon first expanding the images using a series of two-dimensional Prolate Spheroidal Wave Functions (PSWFs), where the expansion coefficients are evaluated using a specially designed numerical integration scheme. Then, the expansion coefficients are used to construct a rotationally-invariant covariance matrix which admits a block-diagonal structure, and the eigen-decomposition of its blocks provides us with the desired steerable principal components. The proposed method is shown to be faster then existing methods, while providing appropriate error bounds which guarantee its accuracy.

arXiv:1605.07254

Convergence guarantees for kernel-based quadrature rules in misspecified settings

Kernel-based quadrature rules are becoming important in machine learning and statistics, as they achieve super-$\sqrt{n}$ convergence rates in numerical integration, and thus provide alternatives to Monte Carlo integration in challenging settings where integrands are expensive to evaluate or where integrands are high dimensional. These rules are based on the assumption that the integrand has a certain degree of smoothness, which is expressed as that the integrand belongs to a certain reproducing kernel Hilbert space (RKHS). However, this assumption can be violated in practice (e.g., when the integrand is a black box function), and no general theory has been established for the convergence of kernel quadratures in such misspecified settings. Our contribution is in proving that kernel quadratures can be consistent even when the integrand does not belong to the assumed RKHS, i.e., when the integrand is less smooth than assumed. Specifically, we derive convergence rates that depend on the (unknown) lesser smoothness of the integrand, where the degree of smoothness is expressed via powers of RKHSs or via Sobolev spaces.

arXiv:1703.04232

Numerical Integration and Dynamic Discretization in Heuristic Search Planning over Hybrid Domains

In this paper we look into the problem of planning over hybrid domains, where change can be both discrete and instantaneous, or continuous over time. In addition, it is required that each state on the trajectory induced by the execution of plans complies with a given set of global constraints. We approach the computation of plans for such domains as the problem of searching over a deterministic state model. In this model, some of the successor states are obtained by solving numerically the so-called initial value problem over a set of ordinary differential equations (ODE) given by the current plan prefix. These equations hold over time intervals whose duration is determined dynamically, according to whether zero crossing events take place for a set of invariant conditions. The resulting planner, FS+, incorporates these features together with effective heuristic guidance. FS+ does not impose any of the syntactic restrictions on process effects often found on the existing literature on Hybrid Planning. A key concept of our approach is that a clear separation is struck between planning and simulation time steps. The former is the time allowed to observe the evolution of a given dynamical system before committing to a future course of action, whilst the later is part of the model of the environment. FS+ is shown to be a robust planner over a diverse set of hybrid domains, taken from the existing literature on hybrid planning and systems.

arXiv:1703.07305

Targeting Bayes factors with direct-path non-equilibrium thermodynamic integration

Thermodynamic integration (TI) for computing marginal likelihoods is based on an inverse annealing path from the prior to the posterior distribution. In many cases, the resulting estimator suffers from high variability, which particularly stems from the prior regime. When comparing complex models with differences in a comparatively small number of parameters, intrinsic errors from sampling fluctuations may outweigh the differences in the log marginal likelihood estimates. In the present article, we propose a thermodynamic integration scheme that directly targets the log Bayes factor. The method is based on a modified annealing path between the posterior distributions of the two models compared, which systematically avoids the high variance prior regime. We combine this scheme with the concept of non-equilibrium TI to minimise discretisation errors from numerical integration. Results obtained on Bayesian regression models applied to standard benchmark data, and a complex hierarchical model applied to biopathway inference, demonstrate a significant reduction in estimator variance over state-of-the-art TI methods.

arXiv:1706.03369

On the Sampling Problem for Kernel Quadrature

The standard Kernel Quadrature method for numerical integration with random point sets (also called Bayesian Monte Carlo) is known to converge in root mean square error at a rate determined by the ratio $s/d$, where $s$ and $d$ encode the smoothness and dimension of the integrand. However, an empirical investigation reveals that the rate constant $C$ is highly sensitive to the distribution of the random points. In contrast to standard Monte Carlo integration, for which optimal importance sampling is well-understood, the sampling distribution that minimises $C$ for Kernel Quadrature does not admit a closed form. This paper argues that the practical choice of sampling distribution is an important open problem. One solution is considered; a novel automatic approach based on adaptive tempering and sequential Monte Carlo. Empirical results demonstrate a dramatic reduction in integration error of up to 4 orders of magnitude can be achieved with the proposed method.

arXiv:1512.00933

Probabilistic Integration: A Role in Statistical Computation?

A research frontier has emerged in scientific computation, wherein numerical error is regarded as a source of epistemic uncertainty that can be modelled. This raises several statistical challenges, including the design of statistical methods that enable the coherent propagation of probabilities through a (possibly deterministic) computational work-flow. This paper examines the case for probabilistic numerical methods in routine statistical computation. Our focus is on numerical integration, where a probabilistic integrator is equipped with a full distribution over its output that reflects the presence of an unknown numerical error. Our main technical contribution is to establish, for the first time, rates of posterior contraction for these methods. These show that probabilistic integrators can in principle enjoy the "best of both worlds", leveraging the sampling efficiency of Monte Carlo methods whilst providing a principled route to assess the impact of numerical error on scientific conclusions. Several substantial applications are provided for illustration and critical evaluation, including examples from statistical modelling, computer graphics and a computer model for an oil reservoir.

arXiv:1802.01610

Fast and accurate approximation of the full conditional for gamma shape parameters

The gamma distribution arises frequently in Bayesian models, but there is not an easy-to-use conjugate prior for the shape parameter of a gamma. This inconvenience is usually dealt with by using either Metropolis-Hastings moves, rejection sampling methods, or numerical integration. However, in models with a large number of shape parameters, these existing methods are slower or more complicated than one would like, making them burdensome in practice. It turns out that the full conditional distribution of the gamma shape parameter is well approximated by a gamma distribution, even for small sample sizes, when the prior on the shape parameter is also a gamma distribution. This article introduces a quick and easy algorithm for finding a gamma distribution that approximates the full conditional distribution of the shape parameter. We empirically demonstrate the speed and accuracy of the approximation across a wide range of conditions. If exactness is required, the approximation can be used as a proposal distribution for Metropolis-Hastings.

arXiv:1803.06989

Numerical Integration on Graphs: where to sample and how to weigh

Let $G=(V,E,w)$ be a finite, connected graph with weighted edges. We are interested in the problem of finding a subset $W \subset V$ of vertices and weights $a_w$ such that $$ \frac{1}{|V|}\sum_{v \in V}^{}{f(v)} \sim \sum_{w \in W}{a_w f(w)}$$ for functions $f:V \rightarrow \mathbb{R}$ that are `smooth' with respect to the geometry of the graph. The main application are problems where $f$ is known to somehow depend on the underlying graph but is expensive to evaluate on even a single vertex. We prove an inequality showing that the integration problem can be rewritten as a geometric problem (`the optimal packing of heat balls'). We discuss how one would construct approximate solutions of the heat ball packing problem; numerical examples demonstrate the efficiency of the method.

arXiv:1705.07079

Scalable Variational Inference for Dynamical Systems

Gradient matching is a promising tool for learning parameters and state dynamics of ordinary differential equations. It is a grid free inference approach, which, for fully observable systems is at times competitive with numerical integration. However, for many real-world applications, only sparse observations are available or even unobserved variables are included in the model description. In these cases most gradient matching methods are difficult to apply or simply do not provide satisfactory results. That is why, despite the high computational cost, numerical integration is still the gold standard in many applications. Using an existing gradient matching approach, we propose a scalable variational inference framework which can infer states and parameters simultaneously, offers computational speedups, improved accuracy and works well even under model misspecifications in a partially observable system.

arXiv:1802.03832공식 코드 · Papers with Code

Quadrature-based features for kernel approximation

We consider the problem of improving kernel approximation via randomized feature maps. These maps arise as Monte Carlo approximation to integral representations of kernel functions and scale up kernel methods for larger datasets. Based on an efficient numerical integration technique, we propose a unifying approach that reinterprets the previous random features methods and extends to better estimates of the kernel approximation. We derive the convergence behaviour and conduct an extensive empirical study that supports our hypothesis.

arXiv:1806.00144

Sea surface temperature prediction and reconstruction using patch-level neural network representations

The forecasting and reconstruction of ocean and atmosphere dynamics from satellite observation time series are key challenges. While model-driven representations remain the classic approaches, data-driven representations become more and more appealing to benefit from available large-scale observation and simulation datasets. In this work we investigate the relevance of recently introduced bilinear residual neural network representations, which mimic numerical integration schemes such as Runge-Kutta, for the forecasting and assimilation of geophysical fields from satellite-derived remote sensing data. As a case-study, we consider satellite-derived Sea Surface Temperature time series off South Africa, which involves intense and complex upper ocean dynamics. Our numerical experiments demonstrate that the proposed patch-level neural-network-based representations outperform other data-driven models, including analog schemes, both in terms of forecasting and missing data interpolation performance with a relative gain up to 50\% for highly dynamic areas.

arXiv:2506.17366v3stat.ML

가우시안 프로세스와 재생핵 힐베르트 공간: 연결점과 동치성

Gaussian Processes and Reproducing Kernel Hilbert Spaces: Connections and Equivalences

본 모노그래프는 양의 정부호 커널(positive definite kernel)을 사용하는 두 가지 접근 방식, 즉 가우시안 프로세스(Gaussian process)를 활용하는 확률론적 방법과 재생핵 힐베르트 공간(RKHS, reproducing kernel Hilbert space)을 활용하는 비확률론적 방법 간의 관계를 연구합니다. 이 두 접근 방식은 머신러닝, 통계학, 수치해석학 분야에서 널리 연구되고 사용됩니다. 본 연구에서는 회귀(regression), 보간(interpolation), 수치 적분(numerical integration), 분포 간 차이(distributional discrepancy), 통계적 의존성(statistical dependence)과 같은 기초적 주제에 대한 연결점과 동치성뿐만 아니라 가우시안 프로세스의 샘플 경로 특성도 다룹니다. 가우시안 힐베르트 공간과 RKHS 간의 동치성을 바탕으로 이러한 동치성에 대한 통합된 관점을 확립합니다. 본 모노그래프는 두 연구 커뮤니티에서 병렬적으로 발전해 온 가우시안 프로세스 및 재생 커널 기반의 수많은 다른 방법들을 잇는 가교 역할을 제공합니다.

arXiv:2607.10793v1cs.LG

Hierarchical Bayesian Quadrature

Numerical integration is a cornerstone of various scientific computing applications, such as engineering simulations and model evidence computations in probabilistic machine learning. Bayesian Quadrature uses Gaussian process surrogates that explicitly encode structural assumptions about the integrand to obtain integral estimates with quantified uncertainty. These surrogates are predominantly based on stationary covariance functions, which results in model misspecification for integrands exhibiting nonstationary behavior. We tackle this issue through an adaptively growing, tree-based partition of the integration domain into local stationary models. Our method recombines the local integral estimates through a hierarchy of GP conditioning that reintroduces cross-subdomain correlations, while model selection criteria control the tree growth to avoid unnecessary partitioning. The resulting algorithm is simple, requires no MCMC, and adapts its evaluation budget to local integrand complexity. On benchmark integration problems and a model evidence computation for an epidemiological model, Hierarchical Bayesian Quadrature achieves substantial gains over standard Bayesian Quadrature on nonstationary integrands while matching its performance on stationary ones.

arXiv:2412.16416v2math.NA

수송 준-몬테카를로

Transport Quasi-Monte Carlo

정규화 흐름(normalizing flows)에서 영감을 받아 단위 초입방체의 균등 분포를 일반적인 목표 분포로 변환하는 수송 맵(transport map)을 학습하여 준-몬테카를로(QMC) 방법을 비정규화 밀도를 가진 복잡한 분포에도 적용할 수 있도록 한다. 제안된 수송 QMC 추정기는 이론적으로 표준 몬테카를로 대비 빠른 수렴 속도를 보장하며, 베이즈 추론 실험에서 그 효과가 확인되었다.

arXiv:2510.01788v2cs.LG

장시간 시뮬레이션을 위한 신경망 기반 비정준 해밀턴 동역학

Neural non-canonical Hamiltonian dynamics for long-time simulations

데이터로부터 비정준 해밀턴 동역학(non-canonical Hamiltonian dynamics)을 학습할 때 발생하는 수치적 불안정성 문제, 특히 퇴화 변분 적분기(Degenerate Variational Integrator)의 게이지 의존성(gauge dependency)으로 인한 장시간 시뮬레이션 실패를 규명하고, 이를 해결하기 위해 벡터 필드 직접 학습과 스킴 기반 시간 이산 동역학 학습이라는 두 가지 학습 전략을 제안한다. Lotka-Volterra, 질량 없는 하전 입자, 자이로운동학적 플라즈마 물리학의 가이딩 센터(guiding center) 등 복잡한 물리 동역학에 대한 수치 실험을 통해 두 방법의 성능을 비교·검증하였다.

arXiv:2111.10722v4stat.ML

적응형 커널을 활용한 최대 평균 불일치 흐름 기반 결정론적 샘플링 방법

A Deterministic Sampling Method via Maximum Mean Discrepancy Flow with Adaptive Kernel

에너지 변분 추론(energetic variational inference) 프레임워크를 활용해 최대 평균 불일치(MMD)를 최소화하는 결정론적 샘플링 방법 EVI-MMD를 제안하며, MMD 최소화 문제를 파티클에 대한 ODE 동적 시스템으로 변환하고 암묵적 오일러 기법으로 해결한다. 핵심 혁신인 동적 대역폭 선택 전략을 통해 커널 기반 방법의 오랜 난제를 개선하였으며, 밀도 함수가 명시된 경우와 학습 데이터만 주어진 생성 모델링 시나리오 모두에서 기존 방법들을 능가함을 수치 실험으로 입증한다.