LIFT: Lifting Language-Controlled Image Fusion to Free-Viewpoint 3D Scenes
원문 초록
Multimodal image fusion traditionally generates a single fused image from paired observations at a specific viewpoint, limiting comprehensive scene perception and adaptability to diverse perceptual objectives. To overcome these limitations, we present LIFT, a novel framework for our proposed language-controlled 3D image fusion (L3D-IF) paradigm that extends multimodal image fusion from 2D images to free-viewpoint 3D scenes, enabling language-guided fusion tailored to diverse perceptual objectives. First, LIFT formulates Semantic Modality Binding (SMB), which interprets language instructions to identify semantic targets and their associated modality preferences and grounds these targets in corresponding regions of a shared Gaussian scene, establishing persistent semantic-to-spatial correspondences for consistent fusion control across viewpoints. Second, Language-Conditioned Gaussian Appearance (LCGA) is developed to achieve scene-wide multimodal appearance modeling by combining adaptive multimodal blending with language- and viewpoint-conditioned appearance refinement, enabling spatially adaptive realization of diverse fusion appearances while preserving source fidelity at pure-modality endpoints. Finally, we construct a Multi-View Language-Supervision (MVLS) dataset that associates language instructions and calibrated viewpoints with modulated fused images, providing effective supervision for learning instruction-dependent fusion from finite multi-view observations. Extensive experiments demonstrate that LIFT achieves coherent multimodal fusion across continuous viewpoints, accurate semantic target localization, and consistent regional fusion control under diverse language instructions.
제출 이력
확인된 버전만 표시합니다. 전체 제출 이력은 arXiv 원문에서 확인하세요.
- arXiv:2611.25696v1제출 시각 미확인
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
공통 태그로 연결된 공개 논문이 아직 없습니다.
토론
이 논문과 연결된 포럼 글이 아직 없습니다.