Biased Stochastic First-Order Methods for Conditional Stochastic Optimization and Applications in Meta Learning
원문 초록
Conditional stochastic optimization covers a variety of applications ranging from invariant learning and causal inference to meta-learning. However, constructing unbiased gradient estimators for such problems is challenging due to the composition structure. As an alternative, we propose a biased stochastic gradient descent (BSGD) algorithm and study the bias-variance tradeoff under different structural assumptions. We establish the sample complexities of BSGD for strongly convex, convex, and weakly convex objectives under smooth and non-smooth conditions. Our lower bound analysis shows that the sample complexities of BSGD cannot be improved for general convex objectives and nonconvex objectives except for smooth nonconvex objectives with Lipschitz continuous gradient estimator. For this special setting, we propose an accelerated algorithm called biased SpiderBoost (BSpiderBoost) that matches the lower bound complexity. We further conduct numerical experiments on invariant logistic regression and model-agnostic meta-learning to illustrate the performance of BSGD and BSpiderBoost.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Imbalanced Regression Pipeline Recommendation
Estimating Interventional Distributions with Uncertain Causal Graphs through Meta-Learning
Causal Inference for Experiments with Latent Outcomes: Key Results and Their Implications for Design and Analysis
Debiased Ill-Posed Regression
Geo-ORBIT: A Federated Digital Twin Framework for Scene-Adaptive Lane Geometry Detection
Active Learning for Manifold Gaussian Process Regression
토론
이 논문과 연결된 포럼 글이 아직 없습니다.