Meta-Learning PAC-Bayes Priors in Model Averaging
원문 초록
Nowadays model uncertainty has become one of the most important problems in both academia and industry. In this paper, we mainly consider the scenario in which we have a common model set used for model averaging instead of selecting a single final model via a model selection procedure to account for this model's uncertainty to improve the reliability and accuracy of inferences. Here one main challenge is to learn the prior over the model set. To tackle this problem, we propose two data-based algorithms to get proper priors for model averaging. One is for meta-learner, the analysts should use historical similar tasks to extract the information about the prior. The other one is for base-learner, a subsampling method is used to deal with the data step by step. Theoretically, an upper bound of risk for our algorithm is presented to guarantee the performance of the worst situation. In practice, both methods perform well in simulations and real data studies, especially with poor-quality data.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
pared: Model selection using multi-objective optimization
Dynamically Learned Test-Time Model Routing in Language Model Zoos with Service Level Guarantees
PEER pressure: Model-to-Model Regularization for Single Source Domain Generalization
Federated Model Search via Reinforcement Learning
Making Language Model a Hierarchical Classifier and Generator
Implementing Adaptations for Vision AutoRegressive Model
토론
이 논문과 연결된 포럼 글이 아직 없습니다.