Towards Robust Out-of-Distribution Generalization Bounds via Sharpness
원문 초록
Generalizing to out-of-distribution (OOD) data or unseen domain, termed OOD generalization, still lacks appropriate theoretical guarantees. Canonical OOD bounds focus on different distance measurements between source and target domains but fail to consider the optimization property of the learned model. As empirically shown in recent work, the sharpness of learned minima influences OOD generalization. To bridge this gap between optimization and OOD generalization, we study the effect of sharpness on how a model tolerates data change in domain shift which is usually captured by "robustness" in generalization. In this paper, we give a rigorous connection between sharpness and robustness, which gives better OOD guarantees for robust algorithms. It also provides a theoretical backing for "flat minima leads to better OOD generalization". Overall, we propose a sharpness-based OOD generalization bound by taking robustness into consideration, resulting in a tighter bound than non-robust guarantees. Our findings are supported by the experiments on a ridge regression model, as well as the experiments on deep learning classification tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
An Information-Theoretic Framework for Out-of-Distribution Generalization with Applications to Stochastic Gradient Langevin Dynamics
Positional Encoding meets Persistent Homology on Graphs
Pruning Spurious Subgraphs for Graph Out-of-Distribtuion Generalization
Learning Where to Learn: Training Distribution Selection for Provable OOD Performance
The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants
LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders
토론
이 논문과 연결된 포럼 글이 아직 없습니다.