Ferret: An Efficient Online Continual Learning Framework under Varying Memory Constraints
원문 초록
In the realm of high-frequency data streams, achieving real-time learning within varying memory constraints is paramount. This paper presents Ferret, a comprehensive framework designed to enhance online accuracy of Online Continual Learning (OCL) algorithms while dynamically adapting to varying memory budgets. Ferret employs a fine-grained pipeline parallelism strategy combined with an iterative gradient compensation algorithm, ensuring seamless handling of high-frequency data with minimal latency, and effectively counteracting the challenge of stale gradients in parallel training. To adapt to varying memory budgets, its automated model partitioning and pipeline planning optimizes performance regardless of memory limitations. Extensive experiments across 20 benchmarks and 5 integrated OCL algorithms show Ferret's remarkable efficiency, achieving up to 3.7$\times$ lower memory overhead to reach the same online accuracy compared to competing methods. Furthermore, Ferret consistently outperforms these methods across diverse memory budgets, underscoring its superior adaptability. These findings position Ferret as a premier solution for efficient and adaptive OCL framework in real-time environments.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis
LifelongPR: Lifelong knowledge fusion for point cloud place recognition based on replay and prompt learning
Rethinking Query-based Transformer for Continual Image Segmentation
Addressing Imbalanced Domain-Incremental Learning through Dual-Balance Collaborative Experts
PSAT: Pediatric Segmentation Approaches via Adult Augmentations and Transfer Learning
Remember Past, Anticipate Future: Learning Continual Multimodal Misinformation Detectors
토론
이 논문과 연결된 포럼 글이 아직 없습니다.