BLISS: Robust Sequence-to-Sequence Learning via Self-Supervised Input Representation
원문 초록
Data augmentations (DA) are the cores to achieving robust sequence-to-sequence learning on various natural language processing (NLP) tasks. However, most of the DA approaches force the decoder to make predictions conditioned on the perturbed input representation, underutilizing supervised information provided by perturbed input. In this work, we propose a framework-level robust sequence-to-sequence learning approach, named BLISS, via self-supervised input representation, which has the great potential to complement the data-level augmentation approaches. The key idea is to supervise the sequence-to-sequence framework with both the \textit{supervised} ("input$\rightarrow$output") and \textit{self-supervised} ("perturbed input$\rightarrow$input") information. We conduct comprehensive experiments to validate the effectiveness of BLISS on various tasks, including machine translation, grammatical error correction, and text summarization. The results show that BLISS outperforms significantly the vanilla Transformer and consistently works well across tasks than the other five contrastive baselines. Extensive analyses reveal that BLISS learns robust representations and rich linguistic knowledge, confirming our claim. Source code will be released upon publication.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LLMs Are Not Scorers: Rethinking MT Evaluation with Generation-Based Methods
Scaling Up Biomedical Vision-Language Models: Fine-Tuning, Instruction Tuning, and Multi-Modal Learning
Twitter Sentiment Analysis
Effective Deep Learning Models for Automatic Diacritization of Arabic Text
APE: Selective Fine-tuning with Acceptance Criteria for Language Model Adaptation
Sentinel: Decoding Context Utilization via Attention Probing for Efficient LLM Context Compression
토론
이 논문과 연결된 포럼 글이 아직 없습니다.