Video Dynamics Prior: An Internal Learning Approach for Robust Video Enhancements
원문 초록
In this paper, we present a novel robust framework for low-level vision tasks, including denoising, object removal, frame interpolation, and super-resolution, that does not require any external training data corpus. Our proposed approach directly learns the weights of neural modules by optimizing over the corrupted test sequence, leveraging the spatio-temporal coherence and internal statistics of videos. Furthermore, we introduce a novel spatial pyramid loss that leverages the property of spatio-temporal patch recurrence in a video across the different scales of the video. This loss enhances robustness to unstructured noise in both the spatial and temporal domains. This further results in our framework being highly robust to degradation in input frames and yields state-of-the-art results on downstream tasks such as denoising, object removal, and frame interpolation. To validate the effectiveness of our approach, we conduct qualitative and quantitative evaluations on standard video datasets such as DAVIS, UCF-101, and VIMEO90K-T.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
EAMamba: Efficient All-Around Vision State Space Model for Image Restoration
Plug-and-Play Linear Attention for Pre-trained Image and Video Restoration Models
Grids Often Outperform Implicit Neural Representations
Exploring Diffusion with Test-Time Training on Efficient Image Restoration
Exploiting the Exact Denoising Posterior Score in Training-Free Guidance of Diffusion Models
Multi-Step Guided Diffusion for Image Restoration on Edge Devices: Toward Lightweight Perception in Embodied AI
토론
이 논문과 연결된 포럼 글이 아직 없습니다.