Out-of-distribution Generalization for Total Variation based Invariant Risk Minimization
원문 초록
Invariant risk minimization is an important general machine learning framework that has recently been interpreted as a total variation model (IRM-TV). However, how to improve out-of-distribution (OOD) generalization in the IRM-TV setting remains unsolved. In this paper, we extend IRM-TV to a Lagrangian multiplier model named OOD-TV-IRM. We find that the autonomous TV penalty hyperparameter is exactly the Lagrangian multiplier. Thus OOD-TV-IRM is essentially a primal-dual optimization model, where the primal optimization minimizes the entire invariant risk and the dual optimization strengthens the TV penalty. The objective is to reach a semi-Nash equilibrium where the balance between the training loss and OOD generalization is maintained. We also develop a convergent primal-dual algorithm that facilitates an adversarial learning scheme. Experimental results show that OOD-TV-IRM outperforms IRM-TV in most situations.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Positional Encoding meets Persistent Homology on Graphs
Pruning Spurious Subgraphs for Graph Out-of-Distribtuion Generalization
The Avengers: A Simple Recipe for Uniting Smaller Language Models to Challenge Proprietary Giants
LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders
On the Out-of-Distribution Generalization of Self-Supervised Learning
From Local Details to Global Context: Advancing Vision-Language Models with Attention-Based Selection
토론
이 논문과 연결된 포럼 글이 아직 없습니다.