Using Image Captions and Multitask Learning for Recommending Query Reformulations
원문 초록
Interactive search sessions often contain multiple queries, where the user submits a reformulated version of the previous query in response to the original results. We aim to enhance the query recommendation experience for a commercial image search engine. Our proposed methodology incorporates current state-of-the-art practices from relevant literature -- the use of generation-based sequence-to-sequence models that capture session context, and a multitask architecture that simultaneously optimizes the ranking of results. We extend this setup by driving the learning of such a model with captions of clicked images as the target, instead of using the subsequent query within the session. Since these captions tend to be linguistically richer, the reformulation mechanism can be seen as assistance to construct more descriptive queries. In addition, via the use of a pairwise loss for the secondary ranking task, we show that the generated reformulations are more diverse.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
SCENIR: Visual Semantic Clarity through Unsupervised Scene Graph Retrieval
Mask-aware Text-to-Image Retrieval: Referring Expression Segmentation Meets Cross-modal Retrieval
A Semantically-Aware Relevance Measure for Content-Based Medical Image Retrieval Evaluation
TNG-CLIP:Training-Time Negation Data Generation for Negation Awareness of CLIP
Multimodal RAG-driven Anomaly Detection and Classification in Laser Powder Bed Fusion using Large Language Models
Generalized Visual Relation Detection with Diffusion Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.