arXiv:2511.06774v2 math.OC

Bilevel Learning via Strategically Inexact Stochastic Gradient Descent

Mohammad Sadegh Salehi, Subhadip Mukherjee, Lindon Roberts, Matthias J. Ehrhardt

제출 2026-11-05

원문 초록

Bilevel optimization is a central tool in computational imaging for high-dimensional hyperparameter tuning; prominent applications include learning data-adaptive regularizers and optimizing forward operators in variational regularization. However, due to the nested nature of the problem, exact gradients with respect to the parameters (so-called hypergradients) are not available: the precision of the hypergradient is linearly related to the computational cost, thus requiring algorithms which can solve the problem without unnecessary precision. In learned-regularizer pipelines, the inner reconstruction solver is typically run for a fixed number of iterations, leaving the hypergradient accuracy uncontrolled and existing convergence theory inapplicable. In this work, we analyze the schedules that practitioners actually run: we prove convergence of inexact stochastic bilevel optimization and establish rates under both decaying accuracy and step size schedules, showing that, with optimal configurations, convergence occurs at an $O(k^{-1/4})$ rate in expectation. This is the optimal rate of nonconvex stochastic gradient descent and matches single-loop stochastic bilevel methods; with warm-started inner solves, total costs also agree up to constants. Experiments on image denoising and inpainting with convex ridge regularizers and input-convex neural networks confirm our analysis: decreasing step sizes improve stability, accuracy scheduling is more critical than step size strategy, and adaptive preconditioning (e.g., Adam) further boosts performance, particularly for input-convex neural networks. In a budget-matched comparison with four state-of-the-art single-loop methods, accuracy-scheduled stochastic gradient descent and its Adam variant attain the highest test peak signal-to-noise ratio.

제출 이력

확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.

  1. arXiv:2511.06774v2제출 시각 미확인

함께 살펴볼 논문

공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.

공통 태그로 연결된 공개 논문이 아직 없습니다.

토론

이 논문과 연결된 포럼 글이 아직 없습니다.