Robustifying Sequential Neural Processes
원문 초록
When tasks change over time, meta-transfer learning seeks to improve the efficiency of learning a new task via both meta-learning and transfer-learning. While the standard attention has been effective in a variety of settings, we question its effectiveness in improving meta-transfer learning since the tasks being learned are dynamic and the amount of context can be substantially smaller. In this paper, using a recently proposed meta-transfer learning model, Sequential Neural Processes (SNP), we first empirically show that it suffers from a similar underfitting problem observed in the functions inferred by Neural Processes. However, we further demonstrate that unlike the meta-learning setting, the standard attention mechanisms are not effective in meta-transfer setting. To resolve, we propose a new attention mechanism, Recurrent Memory Reconstruction (RMR), and demonstrate that providing an imaginary context that is recurrently updated and reconstructed with interaction is crucial in achieving effective attention for meta-transfer learning. Furthermore, incorporating RMR into SNP, we propose Attentive Sequential Neural Processes-RMR (ASNP-RMR) and demonstrate in various tasks that ASNP-RMR significantly outperforms the baselines.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
The Bayesian Approach to Continual Learning: An Overview
Acquiring and Adapting Priors for Novel Tasks via Neural Meta-Architectures
High-Order Deep Meta-Learning with Category-Theoretic Interpretation
The OCR Quest for Generalization: Learning to recognize low-resource alphabets with model editing
Cross-Domain Few-Shot Semantic Segmentation
RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.