Modeling the Machine Learning Multiverse
원문 초록
Amid mounting concern about the reliability and credibility of machine learning research, we present a principled framework for making robust and generalizable claims: the multiverse analysis. Our framework builds upon the multiverse analysis (Steegen et al., 2016) introduced in response to psychology's own reproducibility crisis. To efficiently explore high-dimensional and often continuous ML search spaces, we model the multiverse with a Gaussian Process surrogate and apply Bayesian experimental design. Our framework is designed to facilitate drawing robust scientific conclusions about model performance, and thus our approach focuses on exploration rather than conventional optimization. In the first of two case studies, we investigate disputed claims about the relative merit of adaptive optimizers. Second, we synthesize conflicting research on the effect of learning rate on the large batch training generalization gap. For the machine learning community, the multiverse analysis is a simple and effective technique for identifying robust claims, for increasing transparency, and a step toward improved reproducibility.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition
Assessing parameter identifiability of a hemodynamics PDE model using spectral surrogates and dimension reduction
OmicsQ: A User-Friendly Platform for Interactive Quantitative Omics Data Analysis
A friendly introduction to triangular transport
Kernel Learning Assisted Synthesis Condition Exploration for Ternary Spinel
How causal perspectives can inform problems in computational neuroscience
토론
이 논문과 연결된 포럼 글이 아직 없습니다.