arXiv:2601.22307v3 cs.LG

Exact Gaussian Moment Matching for Residual Networks: a Second-Order Method

Simon Kuang, Xinfan Lin

제출 2026-11-04

원문 초록

We study the problem of propagating the mean and covariance of a general multivariate Gaussian distribution through a deep (residual) neural network using layer-by-layer moment matching. We close a longstanding gap by deriving exact moment matching for the probit, GeLU, ReLU (as a limit of GeLU), Heaviside (as a limit of probit), and sine activation functions; for both feedforward and generalized residual layers. On large-variance random networks, median KL-divergence improvements over common baselines range from tens to thousands of times. On a variational Bayes neural network, we show that our method improves upon a state-of-the-art deterministic inference method by an order of magnitude in KL divergence from Monte Carlo ground truth. We also give a smooth-distance error bound showing that, under regularity assumptions, moment matching removes the leading low-variance errors and propagates higher-order local accuracy through the layers of a network.

제출 이력

확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.

  1. arXiv:2601.22307v3제출 시각 미확인

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

공통 태그로 연결된 공개 논문이 아직 없습니다.

토론

이 논문과 연결된 포럼 글이 아직 없습니다.