Context-PEFT: Efficient Multi-Modal, Multi-Task Fine-Tuning
원문 초록
This paper introduces a novel Parameter-Efficient Fine-Tuning (PEFT) framework for multi-modal, multi-task transfer learning with pre-trained language models. PEFT techniques such as LoRA, BitFit and IA3 have demonstrated comparable performance to full fine-tuning of pre-trained models for specific downstream tasks, all while demanding significantly fewer trainable parameters and reduced GPU memory consumption. However, in the context of multi-modal fine-tuning, the need for architectural modifications or full fine-tuning often becomes apparent. To address this we propose Context-PEFT, which learns different groups of adaptor parameters based on the token's domain. This approach enables LoRA-like weight injection without requiring additional architectural changes. Our method is evaluated on the COCO captioning task, where it outperforms full fine-tuning under similar data constraints while simultaneously offering a substantially more parameter-efficient and computationally economical solution.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
End-to-End Deep Learning of Optimization Heuristics
RaMen: Multi-Strategy Multi-Modal Learning for Bundle Construction
Arctic Inference with Shift Parallelism: Fast and Efficient Open Source Inference System for Enterprise AI
Best Practices for Large-Scale, Pixel-Wise Crop Mapping and Transfer Learning Workflows
Relative Entropy Pathwise Policy Optimization
Robust-Multi-Task Gradient Boosting
토론
이 논문과 연결된 포럼 글이 아직 없습니다.