Exploring Motion-Language Alignment for Text-driven Motion Generation
cs.CV
Submitted: 2026-04-03
Updated: 2026-10-10
License: http://creativecommons.org/licenses/by/4.0/
The gist: Text-driven human motion generation aims to synthesize realistic motion sequences that follow textual descriptions.
Terminology
Abstract
Text-driven human motion generation aims to synthesize realistic motion sequences that follow textual descriptions. Despite recent advances, accurately aligning motion dynamics with textual semantics remains a fundamental challenge. In this paper, we revisit text-to-motion generation from the perspective of motion-language alignment and propose MLA-Gen, a framework that integrates global motion priors with fine-grained local conditioning. This design enables the model to capture common motion patterns, while establishing detailed alignment between texts and motions. Furthermore, we identify a previously overlooked attention sink phenomenon in human motion generation, where attention disproportionately concentrates on the start text token, limiting the utilization of informative textual cues and leading to degraded semantic grounding. To analyze this issue, we introduce SinkRatio, a metric for measuring attention concentration, and develop alignment-aware masking and control strategies to regulate attention during generation. Extensive experiments demonstrate that our approach consistently improves both motion quality and motion-language alignment over strong baselines.
Sources
- Building Normalizing Flows with Stochastic Interpolants
- Why do LLMs attend to the first token?
- CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Models
- CFG-Zero*: Improved Classifier-Free Guidance for Flow Matching Models
- When Attention Sink Emerges in Language Models: An Empirical View
- MoLingo: Motion-Language Alignment for Text-to-Human Motion Generation
- Classifier-Free Diffusion Guidance
- InterMask: 3D Human Interaction Generation via Collaborative Masked Modeling
- MotionPCM: Real-Time Motion Synthesis with Phased Consistency Model
- Auto-Encoding Variational Bayes
- Flow Matching for Generative Modeling
- Flow Matching Guide and Code
- Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow
- Absolute Coordinates Make Motion Generation Easy
- Attention Sinks in Diffusion Language Models
- Rectified-CFG++ for Flow Based Models
- Human Motion Diffusion as a Generative Prior
- Sparse-dLLM: Accelerating Diffusion LLMs with Dynamic Cache Eviction
- Human Motion Diffusion Model
- CFG-Ctrl: Control-Based Classifier-Free Diffusion Guidance
Related papers
- Loss Knows Best: Detecting Annotation Errors in Videos via Loss Trajectories
- AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memories
- Benchmarking the Robustness of Foundation Models for Mammography under Domain Shift
- MambaX-Net: Dual-Input Mamba-Enhanced Cross-Attention Network for Longitudinal MRI Segmentation
- TeleOCR: Navigating Document Parsing Across Digital and Camera-Captured Documents
- A Survey on Efficient Vision-Language-Action Models