Reference-Efficient Multimodal Bitstream-Corrupted Video Recovery via Depth-Conditioned Adaptation
cs.CV
Submitted: 2026-10-11
Updated: 2026-10-14
Comments: 11 pages, 4 figures
Project page: https://intelpro.github.io/UEVD
License: http://creativecommons.org/licenses/by/4.0/
The gist: Recovering visually faithful content from damaged video bitstreams is important for reliable multimedia communication and storage.
Terminology
Abstract
Recovering visually faithful content from damaged video bitstreams is important for reliable multimedia communication and storage. Existing methods exploit local residuals and non-local RGB references of the input video sequence, but both sources can become unreliable after decoding, while processing additional references increases receiver computation and memory. This paper studies multimodal bitstream-corrupted video recovery (MBSCVR) using RGB and depth information, where pre-transmission depth-based geometry aids reconstruction when the received appearance is corrupted. We propose a novel MBSCVR framework that selects and effectively utilizes a single selected non-local RGB reference per recovery window to achieve higher performance. It integrates a reliability-aware multimodal selector that combines valid appearance with depth compatibility before reference encoding. Depth-Conditioned Reference Adaptation (DCRA) introduces selected RGB evidence into feature completion and content generation, while frame-aligned depth conditions completion queries. On the complete BSCV benchmark with oracle masks, MBSCVR uses one selected reference to achieve 32.30 dB on YouTube-VOS and 28.93 dB on DAVIS, surpassing full-reference B2SCVR by 0.27 and 0.33 dB and improving SSIM on both subsets. Matched receiver-forward measurements show 36.8% lower latency and 40.8% lower peak memory. These results demonstrate how compact geometric guidance and adapted appearance processing improve the quality--resource trade-off of multimedia recovery.
Sources
- Bitstream-Corrupted Video Recovery: A Novel Benchmark Dataset and Method
- Towards Blind Bitstream-corrupted Video Recovery via a Visual Foundation Model-driven Framework
- FuseFormer: Fusing Fine-Grained Information in Transformers for Video Inpainting
- ProPainter: Improving Propagation and Transformer for Video Inpainting
- Blind Bitstream-corrupted Video Recovery via Metadata-guided Diffusion Model
- The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
- Depth Anything V2
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models