Consistent Depth of Moving Objects in Video
원문 초록
We present a method to estimate depth of a dynamic scene, containing arbitrary moving objects, from an ordinary video captured with a moving camera. We seek a geometrically and temporally consistent solution to this underconstrained problem: the depth predictions of corresponding points across frames should induce plausible, smooth motion in 3D. We formulate this objective in a new test-time training framework where a depth-prediction CNN is trained in tandem with an auxiliary scene-flow prediction MLP over the entire input video. By recursively unrolling the scene-flow prediction MLP over varying time steps, we compute both short-range scene flow to impose local smooth motion priors directly in 3D, and long-range scene flow to impose multi-view consistency constraints with wide baselines. We demonstrate accurate and temporally coherent results on a variety of challenging videos containing diverse moving objects (pets, people, cars), as well as camera motion. Our depth maps give rise to a number of depth-and-motion aware video editing effects such as object and lighting insertion.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
MonoMVSNet: Monocular Priors Guided Multi-View Stereo Network
Beyond Appearance: Geometric Cues for Robust Video Instance Segmentation
RoboScape: Physics-informed Embodied World Model
RaCalNet: Radar Calibration Network for Sparse-Supervised Metric Depth Estimation
DiFuse-Net: RGB and Dual-Pixel Depth Estimation using Window Bi-directional Parallax Attention and Cross-modal Transfer Learning
Voyager: Long-Range and World-Consistent Video Diffusion for Explorable 3D Scene Generation
토론
이 논문과 연결된 포럼 글이 아직 없습니다.