CRePE: Curved Ray Expectation Positional Encoding for Unified-Camera-Controlled Video Generation

원문 초록

Camera-conditioned video generation requires positional encoding that remains reliable under changes in camera motion, lens configuration, and scene structure. However, existing attention-level camera encodings either provide ray-only camera signals or rely on pinhole camera geometry, limiting their applicability to general camera control under the Unified Camera Model, including wide-angle and fisheye lenses. To address this limitation, we propose Curved Ray Expectation Positional Encoding (CRePE). CRePE represents each image token as a depth-aware positional distribution along its source ray, providing a Unified Camera Model-compatible positional encoding that naturally captures the curved-ray geometry of wide-angle and fisheye cameras. CRePE is implemented through a Geometric Attention Adapter added to frozen video DiTs, injecting token-wise scene-distance information into selected attention layers and stabilizing it with pseudo supervision from a monocular geometry foundation model. This design leads to more stable camera control with competitive video quality in camera-conditioned video generation under the Unified Camera Model, and the resulting model transfers zero-shot to real fisheye and diverse pinhole videos. Controlled positional-encoding ablations show improvements over a RayRoPE-style endpoint approximation, demonstrating the effectiveness of curved-ray-aware positional encoding, particularly for non-pinhole cameras. Furthermore, by extending the same positional-encoding pathway to external geometry control through Radial MixForcing, CRePE supports external radial-map control for scene-geometry-conditioned generation and source-video motion transfer beyond camera control. Project page: https://crepe-neurips2026.github.io/; code: https://github.com/jinotter3/CRePE.