Robustness to Modification with Shared Words in Paraphrase Identification
원문 초록
Revealing the robustness issues of natural language processing models and improving their robustness is important to their performance under difficult situations. In this paper, we study the robustness of paraphrase identification models from a new perspective -- via modification with shared words, and we show that the models have significant robustness issues when facing such modifications. To modify an example consisting of a sentence pair, we either replace some words shared by both sentences or introduce new shared words. We aim to construct a valid new example such that a target model makes a wrong prediction. To find a modification solution, we use beam search constrained by heuristic rules, and we leverage a BERT masked language model for generating substitution words compatible with the context. Experiments show that the performance of the target models has a dramatic drop on the modified examples, thereby revealing the robustness issue. We also show that adversarial training can mitigate this issue.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
NeuralNexus at BEA 2025 Shared Task: Retrieval-Augmented Prompting for Mistake Identification in AI Tutors
HypER: Literature-grounded Hypothesis Generation and Distillation with Provenance
DIBERT: Dependency Injected Bidirectional Encoder Representations from Transformers
A Bit of Progress in Language Modeling
Specification-based Security Evaluation of Real-World Agents Tool-Calling under Long-Horizon Context Tasks
Making Language Model a Hierarchical Classifier and Generator
토론
이 논문과 연결된 포럼 글이 아직 없습니다.