PyTorchKR Papers

최신 논문

총 3,947편
태그·기간 등 상세 필터

카테고리 탐색

로그인 후 이 카테고리 구독

arXiv:2611.18815v1stat.ML

Trajectory-Based Bayesian Discovery of a Host-Pathogen Model from Quasi-Steady-State Observations

System identification of nonlinear dynamical systems becomes challenging when the measured data come from a nearly settled state. We study this situation in the case of a bistable host-pathogen interaction model with a cubic term, whose seven true mechanisms are hidden among twenty candidate terms. The data are from one noise-free trajectory on a quasi-steady-state observed window and the initial state. In this situation, classical sparse identification approaches fail to identify the correct mechanisms as the candidate library becomes ill-conditioned along the quasi-steady-state trajectory. To address this, we propose an alternative approach in which sparse Bayesian learning and $\ell_{0}$ iterative hard thresholding with pursuit provide a good start, and a two-swap climb on the Bayesian evidence of the candidate ODE solutions recovers the true model. Our results show successful model recovery, i.e., correct identification of its support and accurate estimation of its coefficients, as validated by exhaustive scoring.

arXiv:2611.18785v1cs.LG

Setting Optimal Reserve Prices using Fine-Tuned Language Models

Online auction houses must set reserve prices for highly differentiated products whose unstructured descriptions contain valuable demand information. Existing approaches generally require managers to specify product attributes in advance. We ask whether a language model can instead learn from listing descriptions and historical bids to set item-specific reserve prices before bidding begins. We fine-tune a pretrained text embedding language model using the structural likelihood of auction bids as its training objective. This directs representation learning toward the valuation distribution that determines the reserve decision. We show that consecutive upper bids identify the tail information needed to set an optimal reserve without observing or modeling the number of potential bidders. A split-sample estimator learns the representation on one half of the data and conducts structural estimation and inference conditional on that representation on the other. We apply the method to approximately 65,800 online used-vehicle auctions. With seller values calibrated to match the sale rate among listings, the model implies a counterfactual net increase of $165.8 million, representing a 14.6% increase above the incumbent reserve policy. The framework translates a listing description and a seller's value into an item-specific reserve recommendation without constructing a product taxonomy or requiring a bid history for the item being priced. It provides an approach to adapting language models for operational decisions in marketplaces where products are heterogeneous and demand-relevant information is recorded in text.

arXiv:2611.18662v1cs.LG

Stagewise Adaptive Boosting

Stagewise algorithms and sparsity-inducing regularization are two influential approaches to high-dimensional modeling. Existing stagewise theory, however, largely centers on convex regularization, whereas stronger sparsity and other structural preferences are often pursued through fixed-level nonconvex optimization rather than generating an entire adaptive solution path. We develop stagewise adaptive boosting, a general framework that builds a monotone path by selecting the update consistent with structural preference. The framework can encourage sparsity along the path itself and also admits extensions that promote other properties of the solution, including group selection, similarity across models for multiple outcomes, and known priorities or prerequisite relations among predictors. We study its theoretical property as well as finite sample performance using simulations and real-data analyses. Across these studies, the proposed pathwise approach often achieves predictive performance comparable to commonly used baselines while selecting substantially smaller models. These results suggest that adaptive stagewise learning provides a practical route to stronger sparsity and richer structural control without repeatedly solving separate weighted or nonconvex optimization problems.

arXiv:2611.18645v1stat.ML

Domino: Joint Unmasking via First-order Markov Chain for Few-Step Masked Diffusion

Masked diffusion models generate a sequence by revealing masked positions over many denoising steps. A few-step sampler has to reveal many positions at once, but the standard reverse kernel draws them independently and primarily ignores the conditional dependence between tokens revealed in the same step. We propose a novel framework called Domino, which models this dependence with a first-order Markov chain over the revealed positions, given the current state. The first revealed position keeps the posterior of a pre-trained masked diffusion model (MDM), and each subsequent position follows a transition given the token of its predecessor. Notably, the resulting step is normalized by construction, contains the factorized step in MDMs as a special case, and has a likelihood in closed form. We implement the Domino transitions via a lightweight module on the hidden states of a pre-trained MDM denoiser with only 2.6% parameter overhead and train by keeping the MDM denoiser frozen. Our extensive evaluations across text, molecules, and images show that Domino lowers the few-step generative perplexity of MDMs by a factor of 1.7 to 3.8 on One Billion Word (LM1B), WikiText-103, and OpenWebText at nearly unchanged sample entropy. It also raises the validity of few-step molecule samples by 1.5x. The source code is available at: https://github.com/mansoor181/domino.git

arXiv:2611.18625v1stat.ML

Ensemble Bounds for Pairwise Dissimilar Decomposition of Deterministic Datasets

In this paper, we study reduced similarity decompositions of \emph{deterministic} datasets using redundancy graphs and obtain \emph{ensemble} bounds for the minimum size of such constrained decomposition. We consider batch decompositions that reduce similarity \emph{across} batches and use the Lov\'asz local lemma to obtain bounds for the minimum size of a decomposition with a given pairwise dissimilarity. We then consider unique batch decomposition with a given relaxation parameter and obtain upper and lower ensemble bounds for the minimum size. Throughout, we use Binomial random graphs to obtain the desired ensemble bounds, by conditioning on the number of similar \emph{pairs} of data points.

arXiv:2611.17019v2cs.LG

Temporal Regularized Learning: Self-Supervised Learning Local in Space and Time

Self-supervised objectives such as VICReg learn representations by making related inputs similar, while variance and covariance penalties prevent collapse. These penalties are statistics over a batch of samples, so layer-local variants, too, are typically local in space but not in time. We introduce Temporal Regularized Learning (TeReL), which optimizes a VICReg-style objective one sample at a time with purely local updates: each update uses only the current activity of its own and the preceding layer and a fixed-size running state. Temporally adjacent inputs serve as positive pairs, which makes the objective a soft form of Slow Feature Analysis (SFA). Running estimates of each neuron's mean and variance, together with one lateral matrix per layer, replace the batch statistics; held fixed during differentiation, they turn the gradient into a Hebbian feedforward update and an anti-Hebbian lateral update. We show that a two-layer network trained with batch size one on an MNIST stream of class-homogeneous chunks reaches 96.48% linear-probe accuracy, above random features (95.09%) and classical SFA, and that the temporal order and each loss term contribute to this result. With both locality constraints relaxed, the same objective matches supervised backpropagation. TeReL thus shows that the collapse-preventing statistics of modern self-supervised learning can be held as running state at each neuron, which brings these objectives closer to what brains and neuromorphic hardware can implement.

arXiv:2611.16736v2cs.LG

On the Difficulty of Sequence-Level Supervised Learning for Autoregressive Language Models

During the supervised learning phase, autoregressive language models are typically trained by minimizing the negative log-likelihood of the training data, which decomposes as a sum of token-level cross-entropies via teacher forcing. This objective is simple and efficient but potentially suffers from so-called exposure bias: a mismatch between the way the model is trained and the way it is used at inference time. Can we instead design sequence-level loss functions that incorporate inference-time decoding procedures, such as greedy decoding or autoregressive sampling? We show that two natural attempts lead to loss functions that are fundamentally flawed: unbounded below. We further show that these two flawed loss functions are intimately connected: using autoregressive Gumbel tricks, the sampling based loss is the expected greedy loss under Gumbel perturbations. Since the pathology stems from the dependence of the decoded sequence on the model parameters, we analyze the update direction obtained by applying a stop gradient on the sampling procedure, leading to a moment matching direction. While it is not the gradient of any loss function, we prove that it defines a valid descent direction in function space for the autoregressive negative log-likelihood. Despite these theoretical insights, the moment matching direction suffers from high variance. We conclude that teacher forcing may well be ``all you need'' for supervised learning of autoregressive models.

arXiv:2611.16039v2cs.LG

Improved Elastic Weight Consolidation as an Optimization Constraint for Continual Learning

Regularization-based continual learning reduces catastrophic forgetting by penalizing changes to parameters that mattered for earlier tasks. Elastic Weight Consolidation (EWC), a standard instance, weights its quadratic penalty by the empirical Fisher, usually justified as approximating loss curvature. We instead derive the penalty from what training on a new task should preserve. In this formulation, Continual Learning as an Optimization Constraint (CLOC), new training must keep chosen functionals of learned behavior unchanged, and the local approximation of this constraint is the stored penalty. We show that EWC's empirical Fisher is exactly the first-order form of this constraint when the preserved functional is each old sample's loss. The derivation exposes a flaw: EWC weights each sample by the scale of its loss gradient, so samples the model fits well barely contribute, although later training can still forget them. Improved EWC (IEWC) instead penalizes how far each old output moves off its loss level set, rather than how much its loss changes, which removes this weighting for any differentiable loss and output geometry. Across classification, segmentation, language, forecasting, and diffusion, IEWC variants match or improve on EWC and outperform EWC-DR, a classification-specific correction. In a held-out evaluation with validation-selected hyperparameters, removing this weighting alone raises ImageNet-R accuracy over EWC by 2.00 percentage points (95% CI 0.93-3.07). More broadly, CLOC turns the design of continual-learning penalties into a choice of what to preserve, from which a principled penalty follows by derivation.

arXiv:2610.23917v3cs.LG

Announced Breaks Separate Conformal Reliability from Frequency Calibration

When the process behind a time series changes on an announced date, like Australia's switch to five minute electricity settlement on October 1, 2021, intervals calibrated before it can stop covering at their stated level. Whether a forecaster needs the next interval to cover or a finished period's share of misses on target decides the restart. We recompute the next conformal interval from post-break forecast errors, and for n independent ones from the same continuous law, its expected coverage error is $O(n^{-1/2})$, the optimal order in a fixed class among methods told the date. For a finished period's share of misses to match its target, we restart adaptive conformal inference (ACI), which on bounded errors brings that share within $O(n^{-1})$ of target if its corrections at a lower limit stay bounded in total, for that period only. After a simulated mean jump of 5.0 standard deviations, recomputed 90% intervals lie an expected 3.5 percentage points from target at the 50th forecast, a third closer than the 5.2 of the best of five adaptive methods. Of 9 announced changes in Texas, Australian and PJM power markets and a Federal Reserve decision, only Australia's switch moves the errors enough for a 5% test at 80% power to resolve a gain from restarting ACI, and dates help restarted ACI only after large changes.

arXiv:2610.20659v2stat.ML

Latent Annealed Posterior Sampling with a Variance-Preserving Local Gaussian Approximation for Sparsely Observed Random Fields

Posterior inference for random fields from sparse observations presents two major challenges: accurately characterizing complex spatial uncertainty and controlling conditional sampling costs as spatial resolution increases. We propose latent annealed posterior sampling (LAPS). LAPS first maps the random field into a lower dimensional latent space, while a Branch--Trunk coordinate decoder enables field evaluation at arbitrary spatial locations. Flow matching then learns the latent prior distribution. Based on this learned prior, LAPS performs posterior sampling through multilevel annealing. At each level, prior flow transport generates a reference state, which is combined with Gaussian noise using variance-preserving coefficients to construct a local Gaussian approximation of the conditional prior. This approximation is then combined with the observation likelihood to define a local posterior, from which latent posterior samples are generated using preconditioned MALA. Numerical experiments show that LAPS accurately approximates posterior distributions, preserves distinct posterior modes, and decouples posterior conditioning from output resolution. Across two Gaussian benchmarks, LAPS closely matches exact posterior references, with sliced Wasserstein distances $1.8\%$ and $2.4\%$ higher than those obtained from independent exact posterior samples. In the bimodal posterior benchmark, LAPS reduces the sliced Wasserstein distance from $0.2693$ to $0.1499$ and consistently recovers both modes across all test scenarios. For the two dimensional Navier--Stokes random field, LAPS substantially improves marginal posterior calibration while maintaining nearly constant evaluation time as the output grid is refined from $64^2$ to $192^2$. These results demonstrate that LAPS offers a favorable tradeoff among posterior accuracy, uncertainty calibration, and computational efficiency.

arXiv:2610.17053v3stat.ML

Measuring Variable Importance Using Proportional Inclusive Marginal Effects

Shapley-based measures are widely used for measuring variable importance in statistics and machine learning. However, these measures have a serious issue--a nonactive variable can get a positive importance measure when it is correlated with an active variable. Herin et al. recently proposed a measure called proportional marginal effects (PME) that mitigates this spurious effect. However, PME is developed for deterministic models and has high computational cost. In this article, we propose a new measure called proportional inclusive marginal effects (PRIME) that overcomes these two limitations of PME. The key idea in PRIME is to use a novel model-free variable selection technique to identify the active variables and include only them in the computation of the importance measure. Analysis of test functions and real datasets show that PRIME can accurately and reliably quantify variable importance from noisy input-output data.

arXiv:2610.12737v2cs.LG

Kernelized Parameter Sensitivity Atlases for Amortized Conditional Interpretability

Gradient-based attribution provides input-specific information about model parameters, but computing it via backpropagation for many samples is expensive, while static importance scores discard input dependence. We derive KPSA (Kernelized Parameter Sensitivity Atlas) from a block decomposition of the empirical neural tangent kernel. It measures normalized parameter-group sensitivity, embeds those sources in an explicit query space, and uses a lightweight kernel to estimate sensitivities for unseen inputs. We derive bounds that separate finite-source from irreducible representation error and give margin-based top-$k$ recovery. This paper studies two interpretability applications. Firstly, KPSA identifies parameter groups with high susceptibility to perturbations across vision, forecasting, and protein-modeling setups. Across a fivefold range of parameter budgets ($0.02$-$0.1\%$), its selected groups attain $39$-$85\%$ of the perturbation susceptibility of a first-order backpropagation oracle. In vision and forecasting, these parameter groups form sparse sensitivity circuits that extend across depth and vary systematically with the input. Secondly, we use KPSA on an activation-attribution problem for MLP features in transformers. It retains $81\%$ of the gap between a backpropagation oracle and a constant-sensitivity baseline on vision while increasing batched query-time throughput from 551 to 2,041 images/s and reducing peak GPU memory from 4.55 to 0.44GB. On the protein-modeling task, it recovers $68\%$ of this gap. An ablation of KPSA's kernel aggregation shows that nearest-neighbor retrieval in KPSA space can remain competitive. The empirics show that compact kernel atlases can store useful input-dependent parameter sensitivity while amortizing repeated gradient attribution, a step toward scalable conditional interpretability. Code can be found at https://github.com/Axym-Labs/kpsa .

arXiv:2610.10666v2cs.LG

적대적 학습된 신경망의 돌출도 맵 희소성에 대한 규명

Explaining the Saliency Map Sparsity of Adversarially-Trained Neural Networks

심층 신경망이 특정 예측을 수행하는 이유를 이해하는 것은 신경망의 안전한 배포를 위해 매우 중요합니다. 컴퓨터 비전(CV) 분야에서는 예측에 가장 큰 영향을 미치는 이미지 영역을 강조하는 돌출도 맵(saliency map)이 널리 사용되는 설명 방식으로 남아 있습니다. 한 가지 실증적 관찰 결과는 적대적 학습(adversarial training)된 신경망의 그래디언트 돌출도 맵에서 뚜렷한 희소성(sparsity)이 나타난다는 점입니다. 본 논문에서는 2개 층(layer) 렐루(ReLU) 네트워크를 대상으로 이러한 현상에 대한 이론적 설명을 제안합니다. 우리는 특정 손실 함수에서 유효한 것으로 밝혀진 적대적 학습과 가중치 감쇠(weight-decay) 페널티 및 적대적 전변동(total variation) 항이 추가된 경험적 위험 최소화 사이의 기존 동등성 관계를 기반으로 논의를 전개합니다. 데이터 포인트 수와 뉴런 수가 증가하고 정규화 매개변수가 적절한 비율로 0으로 수렴함에 따라, 최소화기(minimizer)가 최소 그래디언트 및 배런 노름(Barron norm)을 갖는 베이즈 분류기(Bayes classifier)로 수렴함을 증명합니다. $\ell_\infty$ 공격을 사용하는 적대적 학습의 경우 그래디언트 노름이 비등방적(anisotropic)이며 축에 정렬된/희소한 그래디언트를 선호하기 때문에 이러한 희소성이 나타납니다. 우리는 자연 학습된 모델과 적대적 학습된 모델의 그래디언트 $\ell_1$-노름 및 임계화된 희소성을 평가하여 이론적 발견을 실험적으로 제시합니다.

arXiv:2610.00793v2stat.ML

Joint neural inference of dynamics and noise covariance for SDEs driven by weighted sub-fractional Brownian motion

We develop a computational procedure for jointly estimating drift, diffusion, temporal covariance weight, and covariance exponent from discrete observations of stochastic differential equations driven by weighted sub-fractional Brownian motion. A change of variables in the Euler recursion yields a trajectory likelihood that retains dependence between Gaussian driving increments and includes the Jacobian associated with state-dependent diffusion. The estimator combines neural-network and radial-basis representations with analytic covariance assembly for piecewise-constant temporal weights. Normalizing the temporal weight separates its global scale from that of diffusion. For each candidate exponent, the optimal global diffusion rescaling is obtained in closed form, leaving a one-dimensional profile search. We compare the procedure with two neural likelihood adaptations on identical simulated trajectories across twenty coefficient settings. Under the specified training schedules, it attains the smallest exponent, diffusion, and temporal-weight errors in 19, 16, and 14 settings, respectively. Drift performance is mixed, with the smallest error in ten settings. These results assess the complete estimation procedures under the prescribed observation design and training schedules.

arXiv:2609.28611v4math.NA

시뮬레이티드 어닐링을 통한 비볼록 최적화에서 3차 랑주뱅 역학의 전역 수렴

Global Convergence of Third-Order Langevin Dynamics for Non-Convex Optimization via Simulated Annealing

We study global convergence guarantees of third-order Langevin dynamics for non-convex optimization via simulated annealing with fixed friction and decreasing noise. An explicit three-block distorted entropy transfers dissipation from the noisy auxiliary variable to the full state. Under dissipativity, regularity, and low-temperature functional-inequality assumptions, logarithmic cooling drives the objective values to the global minimum in probability at the barrier-controlled kinetic rate. For the exact-force-integral and midpoint three-stage discretizations, polynomially decreasing steps preserve this rate on the physical time scale. The cubic local endpoint estimate gives a less restrictive sufficient step-size condition than the available frozen-force kinetic result. A comparison with the one-gradient UBU integrator shows how its centered stochastic local error leads, under the same strong-coupling analysis, to a smaller sufficient iteration exponent. Numerical experiments are conducted to illustrate our theory. For a double well objective, third-order Langevin terminal-success point estimates are higher than UBU at both a common horizon and an equal gradient budget. For a high-dimensional nonconvex neural-network objective using synthetic data, independently tuned UBU and third-order Langevin schemes both outperform overdamped Langevin dynamics; the third-order Langevin point estimate is higher. For the same neural-network objective on real data, we show the same point-estimate ordering for best-basin probability and post-quench test accuracy. Numerical code and associated experiment results are publicly available at https://github.com/gagawjbytw/simulated-annealing-third-order-langevin

arXiv:2609.18682v3stat.ML

DAG ReLU 네트워크의 경로 리프팅 야코비안의 랭크 및 계산

Rank and computation of the pathlifting Jacobian of a DAG ReLU network

본 논문에서는 네트워크의 은닉 노드 수에 대한 귀납법을 수행하여 방향성 비순환 그래프(DAG) 렐루(ReLU) 네트워크의 경로 리프팅(pathlifting) 야코비안(Jacobian)의 랭크(rank)에 대한 자체 완비적인 증명을 제공합니다. 실제로 이 귀납법은 기초적이며, 핵심 비결은 네트워크 경로를 인코딩하는 희소 행렬인 네트워크의 스켈레톤 행렬(skeleton matrix)을 고려하고 은닉 뉴런 중 하나의 표현을 출력 노드로 변환하는 것입니다. 이 증명은 경로 리프팅, 해당 야코비안, 네트워크 매개변수 및 스켈레톤 행렬을 연결하는 중간 명제들에 의존하며, 이는 경로 리프팅 야코비안의 랭크를 도출할 수 있게 할 뿐만 아니라 역전파(backpropagation) 없이 이를 계산하는 방법을 제공하고 실제 계산 비용이 통상적인 역전파에 비해 매우 효율적입니다. 본 논문은 피드포워드 네트워크에 대해 논문의 다양한 명제들을 구현한 Python 모듈을 함께 제공하며, 제안된 이론을 통해 경로 리프팅 야코비안을 계산할 때의 연산상의 이득을 실험적으로 정량화하는 데 사용됩니다.

arXiv:2608.30040v2stat.ML

결측, 측정 오차, 이질성을 공동으로 모델링하는 심층 잠재 변수 프레임워크

A Deep Latent Variable Framework for Jointly Modeling Missingness, Measurement Error, and Heterogeneity

Due to the cost and practical difficulty of obtaining complete and accurate measurements for every individual, population health studies often involve incomplete and error-prone measurements, with missingness and measurement error varying across heterogeneous subpopulations. Although these problems frequently coexist and interact, they are often addressed separately in existing methods. To address these issues, we propose HTree-VAE, a deep latent-variable framework for joint imputation and denoising. The framework combines pattern-aware latent modeling with a hierarchical tree-routed encoder that enables selective parameter sharing across latent subpopulations through a novel reconvergent routing mechanism. It leverages accurate measurements from a validation subset for denoising and explicitly models observation patterns to accommodate both ignorable and nonignorable missingness. Simulation studies demonstrated substantial improvements in reconstruction accuracy over competing deep generative methods. Ablation studies further demonstrated that tree-based and reconvergent routing improved imputation accuracy. We applied HTree-VAE to data from the National Health and Nutrition Examination Survey to denoise self-reported body mass index and impute missing values. HTree-VAE outperformed competing approaches in imputation and achieved competitive performance in correcting self-reported body mass index. These findings highlight the practical value of jointly addressing missingness and measurement error.

arXiv:2608.19584v5cs.LG

K\"ahler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold

We study landscapes for complex-parameterized networks. Our approach is motivated with an information-theoretic manifold perspective of the parameter and via classical optimization guarantees although of complex geometric variety such as through Dolbeault asymptotics. The descent path admits a K\"ahler information metric under a cross-entropy via the Wirtinger Hessian on the log-likelihood potential. We descend according to a differentiated loss scaled by the inverse metric, so the descent path remains in the holomorphic tangent bundle. We emphasize information manifolds which profane theoretical guarantees via ill-curvature-conditioned, or lack thereof, landscapes. We focus on Calabi-Yau metrics specifically in a non-compact setting with a global potential, so defined geometrically rather than invoking the topological requirements of the Calabi conjecture. In non-compact settings, we can write the metric determinant with respect to a background in terms of a function, pluriharmonic or real. Using Yau's Liouville theorem, we get a small eigenvalue blow-up effect under nontrivial background interaction, geometric invariance of the endomorphism constant, and almost low rank assumptions. In an empirical setting, a Ricci-flat metric will not form, but large eigenvalues can be a local condition and hold empirically on open sets even in generalized K\"ahler cases due to lower bounds on the Ricci potential. Moreover, it has been discovered that negative curvature subverts the loss landscape, specifically sectional curvature, so we expand on this and draw interconnections to negative-definite Ricci curvature. Our arguments primarily exist via geometric modality, although we establish roots in deep learning theory such as through asymptotics at initialization and connections through failure modes of neural network guarantees under vanishing and negative Ricci curvature.

arXiv:2608.01654v2math.ST

위상 복원을 위한 무작위 초기화 기반 근사 메시지 전달 기법

Approximate Message Passing with Random Initialization for Phase Retrieval

Randomly initialized first-order iterative algorithms are simple, widely used, and empirically effective across many high-dimensional inference problems, yet their theoretical behavior remains far less understood than that of algorithms started from more elaborate, problem-specific initializations such as spectral initialization. This paper takes a step toward closing this gap by studying randomly initialized approximate message passing (AMP) for noiseless phase retrieval, a canonical nonconvex inference problem. We give a non-asymptotic, growing-horizon characterization of the AMP trajectory from random initialization, and show that it achieves both the information-theoretic weak recovery threshold $\delta_{\rm weak}=1/2$ and the AMP strong recovery threshold $\delta_{\rm str}\approx1.13$, matching the recovery thresholds established for spectrally initialized AMP. The key technical ingredients are twofold: (i) a non-asymptotic coupling that decomposes the AMP iterates as a signal component, a linear combination of independent Gaussian vectors, and an approximation error, and (ii) a second-order analysis of the Onsager correction error built around a non-asymptotic version of Bolthausen's condition. These allow us to analyze the $O_\delta(\log n)$ steps required for weak recovery, and to faithfully track the asymptotic state evolution up to $n^{1/3}/\operatorname{polylog}(n)$ steps in the intermediate recovery regime $\delta\in(\delta_{\rm weak},\delta_{\rm str})$, both beyond the $O(1)$-iteration horizon of classical AMP theory. Above the strong recovery threshold, AMP reaches any prescribed fixed recovery accuracy within $O_{\delta,\varepsilon}(\log n)$ iterations. Both the coupling and the error analysis are developed for a broad class of generalized AMP dynamics and may be of independent interest for general single-index models.

arXiv:2607.16927v2stat.ML

심층 적응형 베이지안 스크리닝

Deep Adaptive Bayesian Screening

We introduce Deep Adaptive Bayesian Screening (DABS), a method for performing adaptive factorial screening in discrete design spaces under tight experimental budgets. DABS learns a policy network offline to sequentially select informative experiments, amortizing Bayesian Optimal Experimental Design. It handles binary designs, incorporates sparsity and interactions via a spike-and-slab prior with strong heredity. The model is trained using a contrastive lower bound on information about factor activity with nuisance effect sizes and noise variance analytically integrated out. Unlike prior amortized Bayesian design approaches, DABS also integrates posterior inference at deployment, yielding posterior probabilities of factor activity and credible intervals on effect sizes. We evaluate DABS on simulated screening problems calibrated to published experimental benchmarks, and show that it matches or outperforms the strongest non-adaptive Bayesian baseline, with the largest gains when the budget is small relative to the number of candidate factors.