Effectiveness of Vision Language Models for Open-world Single Image Test Time Adaptation
원문 초록
We propose a novel framework to address the real-world challenging task of Single Image Test Time Adaptation in an open and dynamic environment. We leverage large scale Vision Language Models like CLIP to enable real time adaptation on a per-image basis without access to source data or ground truth labels. Since the deployed model can also encounter unseen classes in an open world, we first employ a simple and effective Out of Distribution (OOD) detection module to distinguish between weak and strong OOD samples. We propose a novel contrastive learning based objective to enhance the discriminability between weak and strong OOD samples by utilizing small, dynamically updated feature banks. Finally, we also employ a classification objective for adapting the model using the reliable weak OOD samples. The proposed framework ROSITA combines these components, enabling continuous online adaptation of Vision Language Models on a single image basis. Extensive experimentation on diverse domain adaptation benchmarks validates the effectiveness of the proposed framework. Our code can be found at the project site https://manogna-s.github.io/rosita/
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
From Controlled Scenarios to Real-World: Cross-Domain Degradation Pattern Matching for All-in-One Image Restoration
Topology-Aware Modeling for Unsupervised Simulation-to-Reality Point Cloud Recognition
Evidential Spectrum-Aware Contrastive Learning for OOD Detection in Dynamic Graphs
Helping CLIP See Both the Forest and the Trees: A Decomposition and Description Approach
Hierarchical Multi-Positive Contrastive Learning for Patent Image Retrieval
Multiview Contrastive Learning for Unsupervised Domain Adaptation in Brain–Computer Interfaces
토론
이 논문과 연결된 포럼 글이 아직 없습니다.