ED-SAM: An Efficient Diffusion Sampling Approach to Domain Generalization in Vision-Language Foundation Models
원문 초록
The Vision-Language Foundation Model has recently shown outstanding performance in various perception learning tasks. The outstanding performance of the vision-language model mainly relies on large-scale pre-training datasets and different data augmentation techniques. However, the domain generalization problem of the vision-language foundation model needs to be addressed. This problem has limited the generalizability of the vision-language foundation model to unknown data distributions. In this paper, we introduce a new simple but efficient Diffusion Sampling approach to Domain Generalization (ED-SAM) to improve the generalizability of the vision-language foundation model. Our theoretical analysis in this work reveals the critical role and relation of the diffusion model to domain generalization in the vision-language foundation model. Then, based on the insightful analysis, we introduce a new simple yet effective Transport Transformation to diffusion sampling method. It can effectively generate adversarial samples to improve the generalizability of the foundation model against unknown data distributions. The experimental results on different scales of vision-language pre-training datasets, including CC3M, CC12M, and LAION400M, have consistently shown State-of-the-Art performance and scalability of the proposed ED-SAM approach compared to the other recent methods.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
InstructFLIP: Exploring Unified Vision-Language Model for Face Anti-spoofing
Iceberg: Enhancing HLS Modeling with Synthetic Data
Evolution without Large Models: Training Language Model with Task Principles
Thinking in Directivity: Speech Large Language Model for Multi-Talker Directional Speech Recognition
Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models
Alzheimer's Dementia Detection Using Perplexity from Paired Large Language Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.