Physics-Aware Deepfake Detection via Distance-Speech Consistency
cs.CV
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: Interspeech 2026, Project Page: https://byulharang.github.io/PADD/
Journal ref: Proc. Interspeech 2026, pp. 2424-2428 (2026)
DOI: 10.21437/Interspeech.2026-1541
Project page: https://byulharang.github.io/PADD
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recent progress in generative models has made talking-head deepfakes highly realistic and easy to create, jeopardizing public trust.
Terminology
Abstract
Recent progress in generative models has made talking-head deepfakes highly realistic and easy to create, jeopardizing public trust. However, most audio-visual deepfake detectors rely on lip-speech synchronization and are designed for static, frontal videos, limiting their reliability in dynamic, in-the-wild recordings where the speaker moves and lip cues are degraded or missing. As an alternative, we propose a physics-aware detector for dynamic speaking videos that leverages an acoustic constraint: in real recordings, speech energy varies predictably with speaker-to-camera distance, whereas deepfake generation can break this coupling. Our method combines distance estimates from video with distance-dependent acoustic measures from speech to detect deepfakes. Experiments on in-the-wild benchmarks show that the proposed physical cue is effective in dynamic scenarios, and that it complements lip-sync-based detectors, with a simple ensemble achieving consistent gains across datasets. Codes are available at https://byulharang.github.io/PADD/.
Sources
- Movie Gen: A Cast of Media Foundation Models
- Video models are zero-shot learners and reasoners
- SAM Audio: Segment Anything in Audio
- Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024
- Zero-Shot Mono-to-Binaural Speech Synthesis
- Distance-Based Sound Separation
- Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models