Boosting Cross-task Transferability of Adversarial Patches with Visual Relations
원문 초록
The transferability of adversarial examples is a crucial aspect of evaluating the robustness of deep learning systems, particularly in black-box scenarios. Although several methods have been proposed to enhance cross-model transferability, little attention has been paid to the transferability of adversarial examples across different tasks. This issue has become increasingly relevant with the emergence of foundational multi-task AI systems such as Visual ChatGPT, rendering the utility of adversarial samples generated by a single task relatively limited. Furthermore, these systems often entail inferential functions beyond mere recognition-like tasks. To address this gap, we propose a novel Visual Relation-based cross-task Adversarial Patch generation method called VRAP, which aims to evaluate the robustness of various visual tasks, especially those involving visual reasoning, such as Visual Question Answering and Image Captioning. VRAP employs scene graphs to combine object recognition-based deception with predicate-based relations elimination, thereby disrupting the visual reasoning information shared among inferential tasks. Our extensive experiments demonstrate that VRAP significantly surpasses previous methods in terms of black-box transferability across diverse visual reasoning tasks.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Hallucination at a Glance: Controlled Visual Edits and Fine-Grained Multimodal Learning
Multi-Step Visual Reasoning with Visual Tokens Scaling and Verification
MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning
HalLoc: Token-level Localization of Hallucinations for Vision Language Models
An Open-Source Software Toolkit & Benchmark Suite for the Evaluation and Adaptation of Multimodal Action Models
Light as Deception: GPT-driven Natural Relighting Against Vision-Language Pre-training Models
토론
이 논문과 연결된 포럼 글이 아직 없습니다.