Learning Temporally Consistent Video Depth from Video Diffusion Priors
원문 초록
This work addresses the challenge of streamed video depth estimation, which expects not only per-frame accuracy but, more importantly, cross-frame consistency. We argue that sharing contextual information between frames or clips is pivotal in fostering temporal consistency. Thus, instead of directly developing a depth estimator from scratch, we reformulate this predictive task into a conditional generation problem to provide contextual information within a clip and across clips. Specifically, we propose a consistent context-aware training and inference strategy for arbitrarily long videos to provide cross-clip context. We sample independent noise levels for each frame within a clip during training while using a sliding window strategy and initializing overlapping frames with previously predicted frames without adding noise. Moreover, we design an effective training strategy to provide context within a clip. Extensive experimental results validate our design choices and demonstrate the superiority of our approach, dubbed ChronoDepth. Project page: https://xdimlab.github.io/ChronoDepth/.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
BulletGen: Improving 4D Reconstruction with Bullet-Time Generation
Cameras as Relative Positional Encoding
LighthouseGS: Indoor Structure-aware 3D Gaussian Splatting for Panorama-Style Mobile Captures
RoboScape: Physics-informed Embodied World Model
GeoMan: Temporally Consistent Human Geometry Estimation using Image-to-Video Diffusion
From Single Images to Motion Policies via Video-Generation Environment Representations
토론
이 논문과 연결된 포럼 글이 아직 없습니다.