PyTorchKR Papers

업데이트된 논문

총 6편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 태그 구독

arXiv:2602.01929v3math.DS

Function-on-function nonlinear autoregressive model (F2NARX) for probabilistic surrogate modeling of stochastic dynamical systems

Constructing accurate and computationally efficient surrogate models for predicting dynamical system responses is critical in many engineering domains, yet remains challenging due to the strongly nonlinear and high-dimensional mapping from external excitations and system parameters to system responses. This work introduces a novel Function-on-Function Nonlinear AutoRegressive model with eXogenous inputs (F2NARX), which reformulates the recently proposed $\mathcal{F}$-NARX method from a function-on-function regression perspective. The proposed framework substantially improves predictive efficiency while maintaining high accuracy. By combining principal component analysis with Gaussian process regression, F2NARX further enables probabilistic predictions of dynamical responses via the unscented transform in an autoregressive manner. The effectiveness of the method is demonstrated through case studies of varying complexity. Results show that F2NARX outperforms state-of-the-art NARX model by orders of magnitude in efficiency while achieving higher accuracy in general. The probabilistic prediction capability of F2NARX is further demonstrated by combining it with active learning for first-passage probability estimation, showing its potential for reliability-oriented applications.

arXiv:2607.09645v2stat.ML

Deep Gaussian Processes on Directed Acyclic Graphs

Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to multiple fidelity levels; and in gene-regulatory networks, to transcription factors. These functions are partially observed across the DAG, with noisy and heterogeneously sampled measurements, posing significant challenges for reconstruction, uncertainty propagation, and inference. To tackle these challenges, we place priors over functions and naturally arrive at Deep Gaussian Processes over DAGs. We theoretically study their prior-collapse behaviour and the effect of graph topology and intermediate observations on the preservation of information. We obtain almost-sure lower bounds on the asymptotic frequency of depths at which the distinction between inputs is preserved, identify broad kernel classes for which these hold, and prove an observation by \citep{dunlop2018} on the role of input connections. We offer a structured variational approximation that retains graph dependencies, preserves compositional uncertainty, and captures the explaining-away behaviour of colliders. Finally, we empirically validate our theory and methodology at scale and under DAG misspecification. We then model a latent-collider DAG, a protein signalling network, and a multi-fidelity \mbox{heavy-ion} collision emulation task, attaining state-of-the-art performance while recovering low-fidelity contributions and yielding interpretability over the simulator hierarchy.

arXiv:2605.27991v5stat.ML

동적 랜덤 효과 추론으로서의 그래디언트 플로우 최적화: 심층 학습에의 적용을 위한 검정 및 조기 종료

Gradient-Flow Optimization as Dynamic Random-Effects Inference: Testing and Early Stopping with Applications to Deep Learning

그래디언트 플로우 최적화를 랜덤 효과 모델(random-effects model)의 통계적 추론 문제로 재해석하여, 학습 시간을 분산 성분(variance component) 매개변수로 다루는 이론적 프레임워크를 제시한다. 이를 기반으로 REML(restricted maximum likelihood) 기반 조기 종료(early stopping) 규칙을 도출하였으며, UK Biobank 프로테오믹스 데이터 실험에서 검증 데이터 분할 없이도 경쟁력 있는 예측 정확도를 달성함을 보인다.

arXiv:2606.18071v2cs.LG

Volterra Generative Models via Markovian Lifting

Diffusion-based generative models usually perturb data through Markov processes driven by Brownian motion. Volterra processes allow more flexible temporal dependence, but their non-Markovian structure complicates score learning and reverse-time simulation. We develop Volterra generative models and provide Markovian approximations. Gaussian conditioning then reduces the denoising problem to the original data dimension. We present the connections with variance-exploding and variance-preserving representations. Our framework also permits reuse of pretrained denoisers such as EDM2 networks. With one auxiliary factor, our models trained from scratch achieve FIDs of $0.547$ on MNIST and $1.860$ on conditional CIFAR-10. With autoguidance, ImageNet64 FID reaches $1.912$ after fine-tuning, while ImageNet512 FID reaches $1.903$ using frozen pretrained networks.

arXiv:2506.12230v2astro-ph.IM공식 코드 · Papers with Code

Statistical Machine Learning for Astronomy -- A Textbook

This textbook gives a systematic treatment of statistical machine learning for astronomical research, based on Bayesian inference. It shows how modern data analysis techniques connect to classical statistical methods and follow from familiar statistical principles. The Bayesian perspective, used throughout, gives priority to uncertainty quantification and statistical rigor, which scientific inference in astronomy needs. The textbook progresses from probability theory and Bayesian inference through supervised learning, including linear regression with measurement uncertainties, logistic regression and classification. Unsupervised learning topics cover Principal Component Analysis and clustering methods. We then introduce sampling and Markov Chain Monte Carlo, followed by Gaussian Processes as probabilistic nonparametric methods and neural networks in the broader statistical context. Our theory-focused approach derives each method from first principles with complete mathematical development and then shows its astronomical applications. We put statistical insight first and explain why algorithms work, when they are appropriate, and how they connect to broader statistical principles. From a solid grounding in classical methods and their theory, the treatment leads to modern techniques such as neural networks. With this grounding, readers can apply the methods to astronomical research with proper attention to assumptions, limits and uncertainty propagation, as large astronomical surveys require. The 22 companion tutorials apply the methods to astronomical data in Jupyter notebooks.

arXiv:2505.14756v3cs.LG공식 코드 · Papers with Code

LLINBO: Trustworthy LLM-in-the-Loop Bayesian Optimization

Bayesian optimization (BO) is a sequential decision-making tool widely used for optimizing expensive black-box functions. Recently, Large Language Models (LLMs) have shown remarkable adaptability in low-data regimes, making them promising tools for black-box optimization by leveraging contextual knowledge to propose high-quality query points. However, LLMs lack explicit surrogate modeling and calibrated uncertainty, and their internal mechanisms remain opaque. This structural opacity can result in unnecessary experiments, which are costly in BO and may prevent the search from approaching the optimum. To address this, we propose LLINBO: LLM-in-the-Loop BO, a general framework that wraps existing LLM-based BO approaches, deciding when and how to act on their suggestions and when to revert to classical BO. We introduce three mechanisms for this interaction, each accompanied by a no-harm theoretical guarantee that ensures sublinear cumulative regret for common kernels while requiring no performance assumptions on the LLM. Beyond these theoretical guarantees, we show through simulation studies that LLINBO makes existing LLM-based BO more reliable while still exploiting the LLM's few-shot learning ability to outperform classical BO. We further demonstrate its practical use on a real-world, expensive 3D printing experiment. Code is available at https://github.com/UMDataScienceLab/LLM-in-the-Loop-BO.