The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimation
원문 초록
Denoising diffusion probabilistic models have transformed image generation with their impressive fidelity and diversity. We show that they also excel in estimating optical flow and monocular depth, surprisingly, without task-specific architectures and loss functions that are predominant for these tasks. Compared to the point estimates of conventional regression-based methods, diffusion models also enable Monte Carlo inference, e.g., capturing uncertainty and ambiguity in flow and depth. With self-supervised pre-training, the combined use of synthetic and real data for supervised training, and technical innovations (infilling and step-unrolled denoising diffusion training) to handle noisy-incomplete training data, and a simple form of coarse-to-fine refinement, one can train state-of-the-art diffusion models for depth and optical flow estimation. Extensive experiments focus on quantitative performance against benchmarks, ablations, and the model's ability to capture uncertainty and multimodality, and impute missing values. Our model, DDVM (Denoising Diffusion Vision Model), obtains a state-of-the-art relative depth error of 0.074 on the indoor NYU benchmark and an Fl-all outlier rate of 3.26\% on the KITTI optical flow benchmark, about 25\% better than the best published method. For an overview see https://diffusion-vision.github.io.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
fastWDM3D: Fast and Accurate 3D Healthy Tissue Inpainting
Synthesizing Reality: Leveraging the Generative AI-Powered Platform Midjourney for Construction Worker Detection
AdaptaGen: Domain-Specific Image Generation through Hierarchical Semantic Optimization Framework
RobuSTereo: Robust Zero-Shot Stereo Matching under Adverse Weather
Beyond Blur: A Fluid Perspective on Generative Diffusion Models
AI's Blind Spots: Geographic Knowledge and Diversity Deficit in Generated Urban Scenario
토론
이 논문과 연결된 포럼 글이 아직 없습니다.