Fresh-CL: Feature Realignment through Experts on Hypersphere in Continual Learning
원문 초록
Continual Learning enables models to learn and adapt to new tasks while retaining prior knowledge. Introducing new tasks, however, can naturally lead to feature entanglement across tasks, limiting the model's capability to distinguish between new domain data. In this work, we propose a method called Feature Realignment through Experts on hyperSpHere in Continual Learning (Fresh-CL). By leveraging predefined and fixed simplex equiangular tight frame (ETF) classifiers on a hypersphere, our model improves feature separation both intra and inter tasks. However, the projection to a simplex ETF shifts with new tasks, disrupting structured feature representation of previous tasks and degrading performance. Therefore, we propose a dynamic extension of ETF through mixture of experts, enabling adaptive projections onto diverse subspaces to enhance feature representation. Experiments on 11 datasets demonstrate a 2% improvement in accuracy compared to the strongest baseline, particularly in fine-grained datasets, confirming the efficacy of combining ETF and MoE to improve feature distinction in continual learning scenarios.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Growing Transformers: Modular Composition and Layer-wise Expansion on a Frozen Substrate
Little By Little: Continual Learning via Self-Activated Sparse Mixture-of-Rank Adaptive Learning
Breaking Data Silos: Towards Open and Scalable Mobility Foundation Models via Generative Continual Learning
MINGLE: Mixtures of Null-Space Gated Low-Rank Experts for Test-Time Continual Model Merging
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
LifelongPR: Lifelong knowledge fusion for point cloud place recognition based on replay and prompt learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.