Q-Save: Towards Scoring and Attribution for Generated Video Evaluation
cs.CV
Submitted: 2025-11-24
Updated: 2026-10-12
Comments: 24 pages, including references and appendices
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Evaluating AI-generated video (AIGV) quality hinges on three crucial dimensions: visual quality, dynamic quality, and text-video alignment.
Terminology
Abstract
Evaluating AI-generated video (AIGV) quality hinges on three crucial dimensions: visual quality, dynamic quality, and text-video alignment. While numerous evaluation datasets and algorithms have been proposed, existing approaches are constrained by two limitations: the absence of systematic definitions for evaluation dimensions, and the isolated treatment of the three dimensions in separate models. Therefore, we introduce Q-Save, a holistic benchmark dataset and unified evaluation model for AIGV quality assessment. The Q-Save dataset contains nearly 10,000 video samples, each annotated with Mean Opinion Scores (MOS) and fine-grained attribution explanations across the three core dimensions. Leveraging this attribution-annotated dataset, we train the proposed Q-Save model, which adopts the SlowFast framework to balance accuracy and efficiency, and employs a three-stage training strategy with Chain-of-Thought (COT) formatted data: Supervised Fine-Tuning (SFT), Grouped Relative Policy Optimization (GRPO), and a final SFT round for stability, to jointly perform quality scoring and attribution generation. Experimental results demonstrate that Q-Save achieves superior performance in AIGV quality prediction while providing interpretable justifications. Code and dataset will be released upon publication.
Sources
- Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond
- A Note on the Inception Score
- Microsoft COCO Captions: Data Collection and Evaluation Server
- Exploring the Naturalness of AI-Generated Images
- GAIA: Rethinking Action Quality Assessment for AI-Generated Videos
- SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Models
- Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation
- Video-R1: Reinforcing Video Reasoning in MLLMs
- VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation
- VideoScore2: Think before You Score in Generative Video Evaluation
- Prometheus: Inducing Fine-grained Evaluation Capability in Language Models
- HunyuanVideo: A Systematic Framework For Large Video Generative Models
- Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment
- VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluation
- GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation
- AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessment
- Q-Insight: Understanding Image Quality via Visual Reinforcement Learning
- AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment
- Evaluating Text-to-Visual Generation with Image-to-Text Generation
- Improving Video Generation with Human Feedback
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models