When Synthetic Speech Is All You Have: Better Call GRPO

arXiv:2607.08409 · cs.CL, cs.AI · Submitted 2026-07-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "When Synthetic Speech Is All You Have: Better Call GRPO".

Jane: Here's the gist of the paper. The gist The paper shows that Group Relative Policy Optimization (GRPO) extracts far more from synthetic speech than supervised fine-tuning (SFT) because it optimizes a sequence-level objective over sampled hypotheses, which is more robust to localized acoustic artifacts in synthetic speech.

Tom: How it works The core problem addressed is that LLM-based Automatic Speech Recognition (ASR) adapted to regulated domains like banking is bottlenecked by privacy constraints, making synthetic text-to-speech (TTS) an attractive substitute, yet synthetic speech stays acoustically mismatched with real recordings.

Jane: The paper hypothesizes that reliance on SFT caps performance because token-level cross-entropy forces the model to reproduce every reference token by token, absorbing local synthetic artifacts into fluent but acoustically unsupported continuations.

Tom: Group Relative Policy Optimization (GRPO) is introduced as a critic-free method that scores a group of sampled hypotheses and reinforces those that outperform the group using task metrics as the reward function.

Title and authors: Tom: So we're looking at this paper called "When Synthetic Speech Is All You Have: Better Call GRPO." The title tells you right away that they are tackling the situation where you only have synthetic speech, which is common in regulated areas like banking because real data is hard to get.

Jane: Exactly. And the authors are Shashi Kumar, Yanis Labrak, Hasindri Watawana, Sergio Burdisso, Esau Villatoro-Tello, Kadri Hacioglu and Petr Motlicek. They're from institutions like EPFL in Switzerland and Brno University of Technology in Czech Republic.

Lu: The paper is trying to show that Group Relative Policy Optimization or GRPO extracts much more performance from that synthetic speech compared to the standard supervised fine-tuning approach.

Meng: So, what does that mean for us deploying models? Does this mean we can skip collecting massive amounts of real data? Or are we still stuck needing huge banks of actual recordings?

Tom: That’s the question. GRPO is a critic-free method that scores a group of sampled hypotheses and reinforces the ones that do better using task metrics as its reward function. It's focused on optimizing a sequence-level objective over model-generated hypotheses instead of just matching every single reference token, which is what SFT does.

Jane: That sounds complicated, but the core idea seems to be that SFT forces the model to try and reproduce every little piece of text perfectly, even if the synthetic audio has tiny errors. GRPO seems to ignore those local errors and focuses on a better overall transcript.

Lalam: I'm seeing that SFT caps performance because it absorbs those local synthetic artifacts into fluent but acoustically unsupported continuations, according to the paper.

Tom: Right, so SFT is token-level imitation, and GRPO is a discriminative sequence-level adaptation signal that compares complete decoded transcripts. That’s a big shift in how we think about adapting these language models.

The paper's summary: Jane: So, what’s the main finding here? The paper explores the problem that LLM ASR adapted to regulated domains hits a wall because synthetic speech doesn't match real audio well, and SFT just can't bridge that gap effectively.

Lu: They propose GRPO as an alternative, showing it extracts far more from the same synthetic speech than SFT. Specifically, when you only have synthetic data available, GRPO reaches a word error rate of twenty-two point zero nine percent compared to thirty-six point seven one percent for synthetic SFT <ref:2607.08409#pg1>.

Meng: Twenty-two percent on just synthetic banking speech sounds impressive if we’re talking about limited real data availability, but what is that gap actually translating to in terms of real-world deployment reliability?

Tom: The authors trace this gain to behavior rather than representation. GRPO improves stopping calibration and speech-to-text alignment by better anchoring attention to the audio input, which prevents local synthetic defects from turning into global decoding failures.

Jane: So it’s not just about getting a lower error number; it’s about making the model more stable when things get noisy or synthetic. They specifically remove length-independent hallucination by keeping insertion counts low across reference lengths.

Lalam: I'm seeing that GRPO works better because it relies on stable audio evidence, and it avoids continuing when the attention over the audio becomes uninformative, which is a key mechanism there.

The paper's improvements: Tom: The authors suggest a few ways to make this adaptation even stronger. First, they look at how much real data you actually need. They find that the marginal value of real speech is concentrated in the first five to ten hours, where roughly two-thirds of the achievable word error rate reduction comes from less than a fifth of that real data.

Jane: That’s a very concrete number for something like data annotation budgets. It suggests you don't need mountains of real speech to see big gains right off the bat. But they also found that when you fix a small amount of real speech, the relationship with synthetic speech isn't always linear; sometimes more synthetic data doesn't automatically improve things, and around twenty-five hours of synthetic data seems to be the sweet spot.

Lu: Regarding reward functions, they tested WER—which is one minus word error rate—and found that it yields the best percentage word error rate, which is sixteen point seven seven in their experiments. They also looked at character-level rewards and length consistency terms like RLEN, but those combinations didn't beat WER alone for minimizing errors.

Meng: So the recommendation is to just use a word error rate reward and not get bogged down trying to engineer complicated combinations of CER or length terms? That sounds much simpler for our engineering pipeline.

Tom: Exactly. The paper confirms that directly optimizing the target metric, like WER, is sufficient for robust adaptation when you are using GRPO on synthetic data. It's about optimizing the target metric itself rather than adding extra complexity to the reward structure.

Conclusion: Jane: So to wrap up, what does this all mean for the person running an ASR system in a privacy-constrained setting? The main implication is that when synthetic speech is your primary resource, reinforcement learning methods like GRPO are preferred over supervised fine-tuning.

Lu: It’s about recovering most of the performance gap between synthetic and real data using just a few hours of real speech, a word error rate reward function, and a randomly sampled synthetic pool. The gain they found is behavioral rather than representational because GRPO mostly preserves the model's existing representations while making targeted policy corrections.

Meng: From a deployment standpoint, if we are stuck with synthetic data for training or fine-tuning, this paper suggests that using reinforcement learning in this way gives us a much better chance at good performance without needing massive real data budgets.

Tom: So the final practical point is that using WER as a reward is enough and more elaborate combinations of rewards don't help. And when synthetic speech is all you have, GRPO recovers most of the synthetic-to-real gap from a few hours of real speech, a WER reward, and a randomly sampled synthetic pool. That's what the paper "When Synthetic Speech Is All You Have: Better Call GRPO" tells us.

Shashi Kumar, Yanis Labrak, Hasindri Watawana, Sergio Burdisso, Esau Villatoro-Tello, Kadri Hacioglu ˘, Petr Motlicek, Andreas Stolcke

Idiap Research Institute · EPFL University of Lausanne University of Lausanne EPFL University of Lausanne Uniphore U.S.A. · Brno University of Technology Brno Czech Republic

cs.CL, cs.AI

Submitted: 2026-07-09

Updated: 2026-10-20

Comments: 7 pages; accepted to IEEE SLT 2026

License: http://creativecommons.org/licenses/by/4.0/

The gist: The gist The paper shows that Group Relative Policy Optimization (GRPO) extracts far more from synthetic speech than supervised fine-tuning (SFT) because it optimizes a sequence-level objective over

Key concepts

Supervised Fine-Tuning (SFT)
This technique adapts an LLM by training it on paired examples of text and speech. It forces the model to reproduce specific reference tokens exactly, which can cause it to absorb local acoustic errors from synthetic speech into fluent but incorrect continuations.
Group Relative Policy Optimization (GRPO)
A critic-free reinforcement learning method that scores a group of generated hypotheses. It reinforces hypotheses that perform better than the group using task metrics as rewards, shifting adaptation from matching every token to comparing complete decoded transcripts.
Sequence-Level Objective
Instead of focusing on matching individual words (token-level), GRPO optimizes the entire sequence of speech output. This approach is more robust because it focuses on overall transcript quality rather than being overly sensitive to minor, localized acoustic imperfections in synthetic audio.
WER Reward Function
Word Error Rate (WER) is used as the reward function to guide GRPO. The paper found that optimizing the target metric directly—using WER—yielded the best results, confirming that focusing on minimizing errors in the final transcript is sufficient for effective adaptation.

Terminology

Summary

The gist The paper shows that Group Relative Policy Optimization (GRPO) extracts far more from synthetic speech than supervised fine-tuning (SFT) because it optimizes a sequence-level objective over sampled hypotheses, which is more robust to localized acoustic artifacts in synthetic speech.

How it works

The core problem addressed is that LLM-based Automatic Speech Recognition (ASR) adapted to regulated domains like banking is bottlenecked by privacy constraints, making synthetic text-to-speech (TTS) an attractive substitute, yet synthetic speech stays acoustically mismatched with real recordings. The paper hypothesizes that reliance on SFT caps performance because token-level cross-entropy forces the model to reproduce every reference token by token, absorbing local synthetic artifacts into fluent but acoustically unsupported continuations.

Group Relative Policy Optimization (GRPO) is introduced as a critic-free method that scores a group of sampled hypotheses and reinforces those that outperform the group using task metrics as the reward function. It optimizes a sequence-level objective over model-generated hypotheses, shifting adaptation from matching every reference token under the ground-truth prefix to comparing complete decoded transcripts under the model’s own generation distribution. The update is driven by separation between better and worse transcripts rather than token-level imitation alone, providing a discriminative sequence-level adaptation signal.

Comparison of Adaptation Approaches

The study compares GRPO against SFT on both real and synthetic speech data from the LS 960 SFT baseline. On real banking speech, GRPO helps mainly when stacked after SFT, extracting a small additional gain driven primarily by reduced insertions. However, the effect is far larger when only synthetic speech is available, where GRPO reaches 22.09% WER compared to 36.71% for synthetic SFT.

Data Requirements and Reward Functions

The research investigates the necessity of real data, finding that the marginal value of real speech is concentrated in the first 5–10 hours, where roughly two thirds of the achievable WER reduction comes from less than a fifth of the real data. Furthermore, when varying synthetic speech amounts with a fixed 1 h of real speech, the relationship is non-monotonic, suggesting that more synthetic data does not automatically improve performance and that the best setting uses roughly 25 h of synthetic speech. Regarding reward functions, the WER reward yields the best %WER (16.77), confirming the principle of optimizing the target metric directly.

Mechanism of Improvement

GRPO works better than SFT because it addresses insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio. Specifically, GRPO removes length-independent hallucination by keeping insertion counts low across reference lengths, preventing local synthetic defects from propagating into global decoding failures. This is achieved because GRPO relies on stable audio evidence, as it avoids continuing when attention over the audio becomes uninformative.

Conclusion

When synthetic speech is the main or only adaptation resource for LLM-based ASR, reinforcement learning should be preferred over supervised fine-tuning because GRPO recovers most of the synthetic-to-real gap from a few hours of real speech, a WER reward, and a randomly sampled synthetic pool. The gain is behavioral rather than representational because GRPO preserves most representations while acting as a targeted policy correction. The best practical takeaway is that using WER as a reward is enough and more elaborate combinations do not help. No synthetic speech selection strategy consistently beats random sampling once the pool is moderately sized. The overall finding supports the preference for reinforcement learning in privacy-constrained settings.

--- Page 1 ---

When Synthetic Speech Is All You Have: Better Call GRPO.

--- Page 2 ---

Is additional real data necessary? Given a large synthetic speech corpus, how much real speech is worth annotating, and where do returns diminish? (Section IV-B).

--- Page 3 ---

How much synthetic speech is needed when a small amount of data is available? With a tiny real speech budget fixed, how do metrics scale with the amount of synthetic speech? (Section IV-C).

--- Page 4 ---

Which reward function is more optimal? Among WER, CER, a length term, and their combinations, which reward function minimizes errors? (Section IV-D).

--- Page 5 ---

Does selecting a subset of the synthetic training speech help? Do entropy-based selection strategies over speaker, semantic, acoustic embeddings, or GRPO specific selectors such as ranking utterances by rollout statistics, outperform random sampling? (Section IV-E).

--- Page 6 ---

Why does GRPO work better than SFT? Is its advantage behavioral rather than representational? How is the attention anchored to audio inputs after different adaptations approaches? (Section V).

--- Page 7 ---

All code will be released.1 <ref:2607.

Improvements for AI systems

  1. Reinforcement learning based adaptation improves performance when synthetic speech is the main resource by extracting far more from the same synthetic speech than SFT. This allows LLM-based ASR systems to reach a WER of 22.09% on synthetic banking speech alone, significantly closing the gap with SFT's 36.71%.

  2. GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, preventing local synthetic artifacts from propagating into global decoding failures. This results in a lower Expected Calibration Error (ECE) across all utterance lengths, indicating more stable and reliable output.

  3. The system can be optimized using a reward function of WER, as the paper shows it yields the best %WER, confirming that directly optimizing the target metric is sufficient for robust adaptation when using GRPO on synthetic data.

Abstract

LLM-based ASR adapted to regulated domains such as banking is bottlenecked by privacy: real speech is costly and legally constrained to collect, making synthetic text-to-speech (TTS) an attractive substitute. Yet synthetic speech stays acoustically mismatched with real recordings, and work on this gap has stayed within supervised fine-tuning (SFT). We instead turn to reinforcement learning, and show that Group Relative Policy Optimization (GRPO) extracts far more from the same synthetic speech than SFT. Synthetic-only adaptation of the model with GRPO, a critic-free method rewarding low-WER hypotheses, reduces WER by 40% relative to SFT (36.71% to 22.09%), and an SFT-then-GRPO combination pushes this further to 45%. We trace the gain to behavior rather than representation: GRPO reduces insertion errors by improving stopping calibration and speech-to-text alignment by better anchoring attention to audio, leaving early-layer representations intact. When synthetic speech is the main resource, reinforcement learning should be preferred over supervised fine-tuning.

Sources

Related papers