Searching for Effective Multilingual Fine-Tuning Methods: A Case Study in Summarization
원문 초록
Recently, a large number of tuning strategies have been proposed to adapt pre-trained language models to downstream tasks. In this paper, we perform an extensive empirical evaluation of various tuning strategies for multilingual learning, particularly in the context of text summarization. Specifically, we explore the relative advantages of three families of multilingual tuning strategies (a total of five models) and empirically evaluate them for summarization over 45 languages. Experimentally, we not only established a new state-of-the-art on the XL-Sum dataset but also derive a series of observations that hopefully can provide hints for future research on the design of multilingual tuning strategies.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
On-the-Fly Adaptive Distillation of Transformer to Dual-State Linear Attention
Improving large language models with concept-aware fine-tuning
Multimodal Survival Modeling in the Age of Foundation Models
A Split-then-Join Approach to Abstractive Summarization for Very Long Documents in a Low Resource Setting
ConTextual: Improving Clinical Text Summarization in LLMs with Context-preserving Token Filtering and Knowledge Graphs
토론
이 논문과 연결된 포럼 글이 아직 없습니다.