Incrementally Learning Multiple Diverse Data Domains via Multi-Source Dynamic Expansion Model
원문 초록
Continual Learning seeks to develop a model capable of incrementally assimilating new information while retaining prior knowledge. However, current research predominantly addresses a straightforward learning context, wherein all data samples originate from a singular data domain. This paper shifts focus to a more complex and realistic learning environment, characterized by data samples sourced from multiple distinct domains. We tackle this intricate learning challenge by introducing a novel methodology, termed the Multi-Source Dynamic Expansion Model (MSDEM), which leverages various pre-trained models as backbones and progressively establishes new experts based on them to adapt to emerging tasks. Additionally, we propose an innovative dynamic expandable attention mechanism designed to selectively harness knowledge from multiple backbones, thereby accelerating the new task learning. Moreover, we introduce a dynamic graph weight router that strategically reuses all previously acquired parameters and representations for new task learning, maximizing the positive knowledge transfer effect, which further improves generalization performance. We conduct a comprehensive series of experiments, and the empirical findings indicate that our proposed approach achieves state-of-the-art performance.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
PSAT: Pediatric Segmentation Approaches via Adult Augmentations and Transfer Learning
Knowledge Grafting of Large Language Models
The Bayesian Approach to Continual Learning: An Overview
BIRD: Behavior Induction via Representation-structure Distillation
Contrastive Consolidation of Top-Down Modulations Achieves Sparsely Supervised Continual Learning
RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
토론
이 논문과 연결된 포럼 글이 아직 없습니다.