Learning Physical-Spatio-Temporal Features for Video Shadow Removal
원문 초록
Shadow removal in a single image has received increasing attention in recent years. However, removing shadows over dynamic scenes remains largely under-explored. In this paper, we propose the first data-driven video shadow removal model, termed PSTNet, by exploiting three essential characteristics of video shadows, i.e., physical property, spatio relation, and temporal coherence. Specifically, a dedicated physical branch was established to conduct local illumination estimation, which is more applicable for scenes with complex lighting and textures, and then enhance the physical features via a mask-guided attention strategy. Then, we develop a progressive aggregation module to enhance the spatio and temporal characteristics of features maps, and effectively integrate the three kinds of features. Furthermore, to tackle the lack of datasets of paired shadow videos, we synthesize a dataset (SVSRD-85) with aid of the popular game GTAV by controlling the switch of the shadow renderer. Experiments against 9 state-of-the-art models, including image shadow removers and image/video restoration methods, show that our method improves the best SOTA in terms of RMSE error for the shadow area by 14.7. In addition, we develop a lightweight model adaptation strategy to make our synthetic-driven model effective in real world scenes. The visual comparison on the public SBU-TimeLapse dataset verifies the generalization ability of our model in real scenes.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
Plug-and-Play Linear Attention for Pre-trained Image and Video Restoration Models
Zero-to-Hero: Zero-Shot Initialization Empowering Reference-Based Video Appearance Editing
Image Restoration via Multi-domain Learning
Retinex-guided Histogram Transformer for Mask-free Shadow Removal
UDC-VIT: A Real-World Video Dataset for Under-Display Cameras
Discrete Prior-based Temporal-coherent Content Prediction for Blind Face Video Restoration
토론
이 논문과 연결된 포럼 글이 아직 없습니다.