Towards Ambiguity-Free Spatial Foundation Model: Rethinking and Decoupling Depth Ambiguity
원문 초록
Depth ambiguity is a fundamental challenge in spatial scene understanding, especially in transparent scenes where single-depth estimates fail to capture full 3D structure. Existing models, limited to deterministic predictions, overlook real-world multi-layer depth. To address this, we introduce a paradigm shift from single-prediction to multi-hypothesis spatial foundation models. We first present \texttt{MD-3k}, a benchmark exposing depth biases in expert and foundational models through multi-layer spatial relationship labels and new metrics. To resolve depth ambiguity, we propose Laplacian Visual Prompting (LVP), a training-free spectral prompting technique that extracts hidden depth from pre-trained models via Laplacian-transformed RGB inputs. By integrating LVP-inferred depth with standard RGB-based estimates, our approach elicits multi-layer depth without model retraining. Extensive experiments validate the effectiveness of LVP in zero-shot multi-layer depth estimation, unlocking more robust and comprehensive geometry-conditioned visual generation, 3D-grounded spatial reasoning, and temporally consistent video-level depth inference. Our benchmark and code will be available at https://github.com/Xiaohao-Xu/Ambiguity-in-Space.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
OST-Bench: Evaluating the Capabilities of MLLMs in Online Spatio-temporal Scene Understanding
CoNav: Collaborative Cross-Modal Reasoning for Embodied Navigation
EmbRACE-3K: Embodied Reasoning and Action in Complex Environments
ByDeWay: Boost Your multimodal LLM with DEpth prompting in a Training-Free Way
PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly
Learning from Videos for 3D World: Enhancing MLLMs with 3D Vision Geometry Priors
토론
이 논문과 연결된 포럼 글이 아직 없습니다.