A Simple Framework for Open-Vocabulary Zero-Shot Segmentation
원문 초록
Zero-shot classification capabilities naturally arise in models trained within a vision-language contrastive framework. Despite their classification prowess, these models struggle in dense tasks like zero-shot open-vocabulary segmentation. This deficiency is often attributed to the absence of localization cues in captions and the intertwined nature of the learning process, which encompasses both image representation learning and cross-modality alignment. To tackle these issues, we propose SimZSS, a Simple framework for open-vocabulary Zero-Shot Segmentation. The method is founded on two key principles: i) leveraging frozen vision-only models that exhibit spatial awareness while exclusively aligning the text encoder and ii) exploiting the discrete nature of text and linguistic knowledge to pinpoint local concepts within captions. By capitalizing on the quality of the visual representations, our method requires only image-caption pairs datasets and adapts to both small curated and large-scale noisy datasets. When trained on COCO Captions across 8 GPUs, SimZSS achieves state-of-the-art results on 7 out of 8 benchmark datasets in less than 15 minutes.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
AmorLIP: Efficient Language-Image Pretraining via Amortization
Beginning with You: Perceptual-Initialization Improves Vision-Language Representation and Alignment
DEARLi: Decoupled Enhancement of Recognition and Localization for Semi-supervised Panoptic Segmentation
Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Models
GeoVision Labeler: Zero-Shot Geospatial Classification with Vision and Language Models
Uniformity First: Uniformity-aware Test-time Adaptation of Vision-language Models against Image Corruption
토론
이 논문과 연결된 포럼 글이 아직 없습니다.