Open-Vocabulary Object Detection using Pseudo Caption Labels
원문 초록
Recent open-vocabulary detection methods aim to detect novel objects by distilling knowledge from vision-language models (VLMs) trained on a vast amount of image-text pairs. To improve the effectiveness of these methods, researchers have utilized datasets with a large vocabulary that contains a large number of object classes, under the assumption that such data will enable models to extract comprehensive knowledge on the relationships between various objects and better generalize to unseen object classes. In this study, we argue that more fine-grained labels are necessary to extract richer knowledge about novel objects, including object attributes and relationships, in addition to their names. To address this challenge, we propose a simple and effective method named Pseudo Caption Labeling (PCL), which utilizes an image captioning model to generate captions that describe object instances from diverse perspectives. The resulting pseudo caption labels offer dense samples for knowledge distillation. On the LVIS benchmark, our best model trained on the de-duplicated VisualGenome dataset achieves an AP of 34.5 and an APr of 30.6, comparable to the state-of-the-art performance. PCL's simplicity and flexibility are other notable features, as it is a straightforward pre-processing technique that can be used with any image captioning model without imposing any restrictions on model architecture or training process.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Gen-n-Val: Agentic Image Data Generation and Validation
Global Object Proposals for Improving Multi-Sentence Video Descriptions
Towards Reliable Detection of Empty Space: Conditional Marked Point Processes for Object Detection
Multiple Object Stitching for Unsupervised Representation Learning
Domain Randomization for Object Detection in Manufacturing Applications using Synthetic Data: A Comprehensive Study
Edge-Enabled Collaborative Object Detection for Real-Time Multi-Vehicle Perception
토론
이 논문과 연결된 포럼 글이 아직 없습니다.