Time Lens++: Event-based Frame Interpolation with Parametric Non-linear Flow and Multi-scale Fusion
원문 초록
Recently, video frame interpolation using a combination of frame- and event-based cameras has surpassed traditional image-based methods both in terms of performance and memory efficiency. However, current methods still suffer from (i) brittle image-level fusion of complementary interpolation results, that fails in the presence of artifacts in the fused image, (ii) potentially temporally inconsistent and inefficient motion estimation procedures, that run for every inserted frame and (iii) low contrast regions that do not trigger events, and thus cause events-only motion estimation to generate artifacts. Moreover, previous methods were only tested on datasets consisting of planar and faraway scenes, which do not capture the full complexity of the real world. In this work, we address the above problems by introducing multi-scale feature-level fusion and computing one-shot non-linear inter-frame motion from events and images, which can be efficiently sampled for image warping. We also collect the first large-scale events and frames dataset consisting of more than 100 challenging scenes with depth variations, captured with a new experimental setup based on a beamsplitter. We show that our method improves the reconstruction quality by up to 0.2 dB in terms of PSNR and up to 15% in LPIPS score.
함께 살펴볼 논문
공통 태그를 근거로 연결했습니다. 실제 인용 관계나 논문의 우수성을 뜻하지 않습니다.
IBVC: Interpolation-driven B-frame Video Compression
Boost Video Frame Interpolation via Motion Adaptation
Video Frame Interpolation with Densely Queried Bilateral Correlation
BiFormer: Learning Bilateral Motion Estimation via Bilateral Transformer for 4K Video Frame Interpolation
EventDiff: A Unified and Efficient Diffusion Model Framework for Event-based Video Frame Interpolation
TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion
토론
이 논문과 연결된 포럼 글이 아직 없습니다.