Gen Through the Lens: Camera-Controllable Video Generation via Cross-Coordinate Conditioning Fusion
cs.CV, cs.AI, cs.GR, cs.LG
Submitted: 2026-10-11
Updated: 2026-10-11
Comments: 15 pages, 9 figures, 5 tables. Shuai Li and Jinxu Xiang contributed equally; Ran Zhang is the corresponding author
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: We present Gen Through the Lens, a camera-controllable image-to-video generation framework that synthesizes videos from a given reference image while enabling realistic and physically consistent
Abstract
We present Gen Through the Lens, a camera-controllable image-to-video generation framework that synthesizes videos from a given reference image while enabling realistic and physically consistent camera operations. Unlike previous models that treat camera motion as abstract conditioning or rely on a single coordinate system, our approach explicitly decomposes and fuses control signals across three coordinate spaces: the camera coordinate (modeled by intrinsic parameters), the world coordinate (represented by spatial geometry hints), and the object coordinate (described by 3D bounding boxes of target subjects). Through cross-coordinate conditioning fusion, these heterogeneous cues are unified to guide the generative process, enabling precise control of camera motion, focus, and framing while preserving temporal and spatial coherence. Extensive experiments demonstrate that our method produces controllable and visually consistent results across diverse camera operations, outperforming existing approaches in both realism and editability.
Sources
- ReCamMaster: Camera-Controlled Generative Rendering from A Single Video
- Perception-as-Control: Fine-grained Controllable Image Animation with 3D-aware Motion Representation
- Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scaling
- I2VControl-Camera: Precise Video Camera Control with Adjustable Motion Strength
- Motion Prompting: Controlling Video Generation with Motion Trajectories
- AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning
- CameraCtrl: Enabling Camera Control for Text-to-Video Generation
- VACE: All-in-One Video Creation and Editing
- MapAnything: Universal Feed-Forward Metric 3D Reconstruction
- AnimateAnything: Consistent and Controllable Animation for Video Generation
- SAM 2: Segment Anything in Images and Videos
- Seedance 2.0: Advancing Video Generation for World Complexity
- BEDLAM2.0: Synthetic Humans and Cameras in Motion
- Wan: Open and Advanced Large-Scale Video Generative Models
- Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution
- MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Details
- $\pi^3$: Permutation-Equivariant Visual Geometry Learning
- DragAnything: Motion Control for Anything using Entity Representation
- CamCo: Camera-Controllable 3D-Consistent Image-to-Video Generation
- DragNUWA: Fine-grained Control in Video Generation by Integrating Text, Image, and Trajectory
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models