The previous fully-supervised and weakly-supervised camouflaged object segmentation tasks required a significant amount of annotated data for supervised training to enable models to segment camouflaged objects effectively. However, models like the Segment Anything Model (SAM), which falls under the category of Promptable Segmentation models, can achieve excellent segmentation performance on unseen images with just an instance-specific visual prompt. Nevertheless, for complex scenarios like camouflaged objects, SAM may not perform well even with an instance-specific prompt. Furthermore, the question arises: Is an instance-specific prompt necessary? In more realistic scenarios, where only a task-generic task description is provided as a universally applicable text prompt, how can we improve segmentation across various datasets under the camouflaged object segmentation task?
The Segment Anything Model (SAM) shows remarkable segmentation ability with sparse prompts like points. However, manual prompt is not always feasible, as it may not be accessible in real-world application. In this work, we aim to eliminate the need for manual prompt.The key idea is to employ Cross-modal Chains of Thought Prompting (CCTP) to reason visual prompts using the semantic information given by a generic text prompt. We introduce a test-time adaptation per-instance mechanism called Generalizable SAM (GenSAM) to automatically generate and optimize visual prompts the generic task prompt. CCTP maps a single generic text prompt onto image-specific consensus foreground and background heatmaps using vision-language models, acquiring reliable visual prompts. Moreover, to test-time adapt the visual prompts, we further propose Progressive Mask Generation (PMG) to iteratively reweight the input image, guiding the model to focus on the targets in a coarse-to-fine manner.Crucially, all network parameters are fixed, avoiding the need for additional training.Experiments demonstrate the superiority of GenSAM. Experiments on three benchmarks demonstrate that GenSAM outperforms point supervision approaches and achieves comparable results to scribble supervision ones, solely relying on general task descriptions as prompts.