Relationship-based Neural Baby Talk
원문 초록
Understanding interactions between objects in an image is an important element for generating captions. In this paper, we propose a relationship-based neural baby talk (R-NBT) model to comprehensively investigate several types of pairwise object interactions by encoding each image via three different relationship-based graph attention networks (GATs). We study three main relationships: \textit{spatial relationships} to explore geometric interactions, \textit{semantic relationships} to extract semantic interactions, and \textit{implicit relationships} to capture hidden information that could not be modelled explicitly as above. We construct three relationship graphs with the objects in an image as nodes, and the mutual relationships of pairwise objects as edges. By exploring features of neighbouring regions individually via GATs, we integrate different types of relationships into visual features of each node. Experiments on COCO dataset show that our proposed R-NBT model outperforms state-of-the-art models trained on COCO dataset in three image caption generation tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Heterogeneous Graph Attention Network With Multimodal Feature Selection for Cancer Multiomics Integration
Following the Clues: Experiments on Person Re-ID using Cross-Modal Intelligence
DenseWorld-1M: Towards Detailed Dense Grounded Caption in the Real World
SonicVerse: Multi-Task Learning for Music Feature-Informed Captioning
AST-Enhanced or AST-Overloaded? The Surprising Impact of Hybrid Graph Representations on Code Clone Detection
STAMImputer: Spatio-Temporal Attention MoE for Traffic Data Imputation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.