Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Model
원문 초록
Cross-attention is an important component of neural machine translation (NMT), which is always realized by dot-product attention in previous methods. However, dot-product attention only considers the pair-wise correlation between words, resulting in dispersion when dealing with long sentences and neglect of source neighboring relationships. Inspired by linguistics, the above issues are caused by ignoring a type of cross-attention, called concentrated attention, which focuses on several central words and then spreads around them. In this work, we apply Gaussian Mixture Model (GMM) to model the concentrated attention in cross-attention. Experiments and analyses we conducted on three datasets show that the proposed method outperforms the baseline and has significant improvement on alignment quality, N-gram accuracy, and long sentence translation.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT
Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation
Do Not Change Me: On Transferring Entities Without Modification in Neural Machine Translation -- a Multilingual Perspective
Intrinsic vs. Extrinsic Evaluation of Czech Sentence Embeddings: Semantic Relevance Doesn't Help with MT Evaluation
Building a Functional Machine Translation Corpus for Kpelle
In-Domain African Languages Translation Using LLMs and Multi-armed Bandits
토론
이 논문과 연결된 포럼 글이 아직 없습니다.