Pitfalls in the Evaluation of Sentence Embeddings
원문 초록
Deep learning models continuously break new records across different NLP tasks. At the same time, their success exposes weaknesses of model evaluation. Here, we compile several key pitfalls of evaluation of sentence embeddings, a currently very popular NLP paradigm. These pitfalls include the comparison of embeddings of different sizes, normalization of embeddings, and the low (and diverging) correlations between transfer and probing tasks. Our motivation is to challenge the current evaluation of sentence embeddings and to provide an easy-to-access reference for future research. Based on our insights, we also recommend better practices for better future evaluations of sentence embeddings.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
LLMs Are Not Scorers: Rethinking MT Evaluation with Generation-Based Methods
Contrastive Prompting Enhances Sentence Embeddings in LLMs through Inference-Time Steering
Sentence Embeddings as an intermediate target in end-to-end summarisation
JTCSE: Joint Tensor-Modulus Constraints and Cross-Attention for Unsupervised Contrastive Learning of Sentence Embeddings
Information Leakage of Sentence Embeddings via Generative Embedding Inversion Attacks
Domain Adaptation for Japanese Sentence Embeddings with Contrastive Learning based on Synthetic Sentence Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.