TACOS: Task Agnostic Continual Learning in Spiking Neural Networks
원문 초록
Catastrophic interference, the loss of previously learned information when learning new information, remains a major challenge in machine learning. Since living organisms do not seem to suffer from this problem, researchers have taken inspiration from biology to improve memory retention in artificial intelligence systems. However, previous attempts to use bio-inspired mechanisms have typically resulted in systems that rely on task boundary information during training and/or explicit task identification during inference, information that is not available in real-world scenarios. Here, we show that neuro-inspired mechanisms such as synaptic consolidation and metaplasticity can mitigate catastrophic interference in a spiking neural network, using only synapse-local information, with no need for task awareness, and with a fixed memory size that does not need to be increased when training on new tasks. Our model, TACOS, combines neuromodulation with complex synaptic dynamics to enable new learning while protecting previous information. We evaluate TACOS on sequential image recognition tasks and demonstrate its effectiveness in reducing catastrophic interference. Our results show that TACOS outperforms existing regularization techniques in domain-incremental learning scenarios. We also report the results of an ablation study to elucidate the contribution of each neuro-inspired mechanism separately.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Experts
CL-LoRA: Continual Low-Rank Adaptation for Rehearsal-Free Class-Incremental Learning
PrePrompt: Predictive prompting for class incremental learning
LoRAX: LoRA eXpandable Networks for Continual Synthetic Image Attribution
IAP: Improving Continual Learning of Vision-Language Models via Instance-Aware Prompting
Do Your Best and Get Enough Rest for Continual Learning
토론
이 논문과 연결된 포럼 글이 아직 없습니다.