SparTerm: Learning Term-based Sparse Representation for Fast Text Retrieval
원문 초록
Term-based sparse representations dominate the first-stage text retrieval in industrial applications, due to its advantage in efficiency, interpretability, and exact term matching. In this paper, we study the problem of transferring the deep knowledge of the pre-trained language model (PLM) to Term-based Sparse representations, aiming to improve the representation capacity of bag-of-words(BoW) method for semantic-level matching, while still keeping its advantages. Specifically, we propose a novel framework SparTerm to directly learn sparse text representations in the full vocabulary space. The proposed SparTerm comprises an importance predictor to predict the importance for each term in the vocabulary, and a gating controller to control the term activation. These two modules cooperatively ensure the sparsity and flexibility of the final text representation, which unifies the term-weighting and expansion in the same framework. Evaluated on MSMARCO dataset, SparTerm significantly outperforms traditional sparse methods and achieves state of the art ranking performance among all the PLM-based sparse models.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Kodezi Chronos: A Debugging-First Language Model for Repository-Scale, Memory-Driven Code Understanding
Differential Mamba
RMIT-ADM+S at the SIGIR 2025 LiveRAG Challenge
FlexRAG: A Flexible and Comprehensive Framework for Retrieval-Augmented Generation
XGraphRAG: Interactive Visual Analysis for Graph-based Retrieval-Augmented Generation
Llama Nemoretriever Colembed: Top-Performing Text-Image Retrieval Model
토론
이 논문과 연결된 포럼 글이 아직 없습니다.