GestureFAR: Streaming Co-Speech Gesture Generation with Flow Autoregression

arXiv:2609.21576 · cs.CV, cs.GR, cs.HC · Submitted 2026-09-18 · Read on arXiv

cs.CV, cs.GR, cs.HC

Submitted: 2026-09-18

Updated: 2026-09-18

Project page: https://andypinxinliu.github.io/GestureFAR

License: http://creativecommons.org/licenses/by/4.0/

The gist: Generating natural co-speech gestures from streaming speech is essential for embodied conversational agents, where motion must be produced while a user is still speaking.

Terminology

Abstract

Generating natural co-speech gestures from streaming speech is essential for embodied conversational agents, where motion must be produced while a user is still speaking. Recent streaming gesture systems make online generation possible by autoregressing over discrete motion tokens, but this design compresses high-dimensional continuous motion into finite codebooks and can limit the realism and diversity of generated gestures. To preserve both causality and continuous expressiveness, we propose GestureFAR, a flow-autoregressive framework for streaming co-speech gesture generation. First, GestureFAR autoregresses over causal continuous motion latents, using a transformer to model streaming audio-motion context and a per-token flow-matching head to sample the next latent from a continuous distribution. Second, we introduce a head-only flow distillation strategy that freezes the causal backbone and distills the multi-step per-token flow head into a single network evaluation using consistency and distribution-matching objectives. This keeps the model token-causal while removing the main latency bottleneck for live interaction. Experiments on BEAT2 show that GestureFAR significantly improves the quality--latency trade-off among streaming-capable methods, preserving strong gesture quality while enabling real-time token-causal generation. Project Page: https://andypinxinliu.github.io/GestureFAR

Sources

Related papers