CoT-MAE v2: Contextual Masked Auto-Encoder with Multi-view Modeling for Passage Retrieval
원문 초록
Growing techniques have been emerging to improve the performance of passage retrieval. As an effective representation bottleneck pretraining technique, the contextual masked auto-encoder utilizes contextual embedding to assist in the reconstruction of passages. However, it only uses a single auto-encoding pre-task for dense representation pre-training. This study brings multi-view modeling to the contextual masked auto-encoder. Firstly, multi-view representation utilizes both dense and sparse vectors as multi-view representations, aiming to capture sentence semantics from different aspects. Moreover, multiview decoding paradigm utilizes both autoencoding and auto-regressive decoders in representation bottleneck pre-training, aiming to provide both reconstructive and generative signals for better contextual representation pretraining. We refer to this multi-view pretraining method as CoT-MAE v2. Through extensive experiments, we show that CoT-MAE v2 is effective and robust on large-scale passage retrieval benchmarks and out-of-domain zero-shot benchmarks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Enhancing Automatic Term Extraction with Large Language Models via Syntactic Retrieval
Multimodal Representation Alignment for Cross-modal Information Retrieval
Interpretable Natural Language Segmentation Based on Link Grammar
Hierarchical Gumbel Attention Network for Text-based Person Search
W2VV++: Fully Deep Learning for Ad-hoc Video Search
BIOSSES: A Semantic Sentence Similarity Estimation System for the Biomedical Domain
토론
이 논문과 연결된 포럼 글이 아직 없습니다.