Exploring Fine-tuning Techniques for Pre-trained Cross-lingual Models via Continual Learning
원문 초록
Recently, fine-tuning pre-trained language models (e.g., multilingual BERT) to downstream cross-lingual tasks has shown promising results. However, the fine-tuning process inevitably changes the parameters of the pre-trained model and weakens its cross-lingual ability, which leads to sub-optimal performance. To alleviate this problem, we leverage continual learning to preserve the original cross-lingual ability of the pre-trained model when we fine-tune it to downstream tasks. The experimental result shows that our fine-tuning methods can better preserve the cross-lingual ability of the pre-trained model in a sentence retrieval task. Our methods also achieve better performance than other fine-tuning baselines on the zero-shot cross-lingual part-of-speech tagging and named entity recognition tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Selecting and Merging: Towards Adaptable and Scalable Named Entity Recognition with Large Language Models
Flippi: End To End GenAI Assistant for E-Commerce
Better Semi-supervised Learning for Multi-domain ASR Through Incremental Retraining and Data Filtering
Efficient Data Selection for Domain Adaptation of ASR Using Pseudo-Labels and Multi-Stage Filtering
Transformer-Based Named Entity Recognition for Automated Server Provisioning
HEProto: A Hierarchical Enhancing ProtoNet based on Multi-Task Learning for Few-shot Named Entity Recognition
토론
이 논문과 연결된 포럼 글이 아직 없습니다.