Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition".

Jane: Here's the gist of the paper. The gist: Reverse multi-stage fine-tuning (R-MFT) enables a parameter-efficient 244M Whisper model to match or exceed conventionally fine-tuned 769M counterparts for spontaneous Indic speech by employing a high initial learning rate and a hard-to-easy curriculum.

Tom: Vividh-ASR Benchmark Vividh-ASR is introduced as a complexity-stratified benchmark designed to diagnose studio-bias in Indic speech recognition It stratifies evaluation by acoustic complexity across four tiers: studio (Tier A), broadcast (Tier B), spontaneous (Tier C), and synthetic noise (Tier D) Tier A consists of "Scripted, read speech in controlled environments" Tier B involves "Read speech from news broadcasts" Tier C represents "Crowdsourced, unscripted recordings with disfluencies, varying prosody, background noise, and non-professional hardware" Tier D is defined as "Tier A audio augmented with synthetic noise profiles (babble, music, environmental)" The data distribution is intentionally weighted toward Tier C, reflecting the focus on closing the spontaneous-speech performance gap.

Jane: Optimization Dynamics and Curriculum Study The authors systematically decoupled the effects of learning-rate timing and curriculum direction through a controlled 2×2 factorial study The two axes tested were Learning Rate (LR) Timing, comparing a decreasing schedule against an increasing schedule, and Curriculum Direction, comparing an easy-to-hard ordering against a hard-to-easy ordering.

Tom: Key findings from this study indicate that "early large parameter updates improve global WER by ∼12 absolute points" while a "hard-to-easy curriculum adds gains for spontaneous speech".

Title and authors: Jane: So, we're looking at this paper now called Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition. Tom, what's the basic idea here for a listener who isn't deep in machine learning?

Tom: Basically, they’re talking about how fine-tuning big speech models like Whisper works when you try to use them for languages that aren't super common. The authors point out that if you just fine-tune on read speech, it actually makes the model worse when it tries to understand spontaneous speech, which is how people actually talk in real life.

Lu: They introduce this benchmark called Vividh-ASR to diagnose exactly where models fail by stratifying evaluation across four tiers: studio, broadcast, spontaneous, and synthetic noise. It's meant to isolate the problem of studio bias in Indic speech recognition across these different levels of acoustic difficulty <ref:2605.13087#pg1>.

Meng: So it’s not just one test; it breaks down performance based on how noisy or complex the audio actually is—from controlled recordings to real crowds with background noise <ref:2605.13087#pg1>. It lets them see if a model works for casual conversation or if it only works well in perfect studio settings.

Lalam: I see this as a way to stop people from just assuming that because a model does okay on read speech, it's actually ready for spontaneous audio too <ref:2605.13087#pg2>.

The paper's summary: Tom: The core finding they present is that simply using a standard fine-tuning approach doesn't fix this mismatch; you need to change *how* you tune the model. They studied learning rate timing and curriculum ordering to see what actually helps.

Jane: What they found was pretty concrete: starting with a high learning rate helps improve the overall Word Error Rate by about twelve absolute points <ref:2605.13087#pg1> <ref:2605.13087#pg1>. And they also found that a hard-to-easy order for the data actually adds gains specifically for spontaneous speech <ref:2605.13087#pg1>.

Lu: This leads them to propose something called Reverse Multi-Stage Fine-Tuning, or R-MFT, which is a training recipe designed to be parameter efficient. It uses three stages and a specific learning rate schedule that pairs the best conditions together <ref:2605.13087#pg3>.

Meng: From an engineering standpoint, they are trying to get a smaller two hundred forty-four million Whisper model to perform better than a much larger seven hundred sixty-nine million baseline by using this R-MFT recipe <ref:2605.13087#pg1>. That parameter efficiency is what’s really interesting for us at scale.

Lalam: It seems like the idea is that you need to break out of the model's initial pre-trained state with a big push before you can get it to learn the harder stuff <ref:2605.13087#pg2>.

The paper's improvements: Tom: So what are they suggesting we actually *do* differently? They are suggesting this R-MFT recipe, which is essentially pairing a high initial learning rate with a hard-to-easy curriculum for the first stage of training <ref:2605.13087#pg3>.

Jane: Think about it this way: you start aggressive to get the model moving past its starting point, then you move through different data difficulties in an order that gradually gets harder <ref:2605.13087#pg1>. It’s about controlling the learning process itself rather than just throwing more data at it randomly.

Lu: The paper also shows how this works internally using Centered Kernel Alignment and Singular Value Decomposition to prove that these effective schedules concentrate the adaptation in the decoder part of the model while keeping the encoder's acoustic geometry stable <ref:2605.13087#pg1>.

Meng: That’s a big deal for deployment because it means we aren't destroying all the knowledge in the core encoder; we’re just adjusting what processes audio into text, which makes it much more efficient to fine-tune and maintain.

Lalam: So the improvement is not just about getting better scores on a test set; it’s about finding a training path that respects how speech models are built internally <ref:2605.13087#pg1>.

Conclusion: Jane: So, to wrap up on Vividh-ASR, the main implication is that for Indic ASR, raw model scaling isn't always the answer; controlling the optimization trajectory with R-MFT can actually outperform simply using a bigger model <ref:2605.13087#pg1>.

Tom: The key thing to remember is that aggressive early learning rates are dominant; they yield about twelve absolute points in Word Error Rate improvement, and later high-energy updates don't help if you start too conservatively <ref:2605.13087#pg3> <ref:2605.13087#pg1>.

Lu: It confirms that a hard-data first optimization trajectory is more effective than raw model scaling for getting robust Indic ASR results <ref:2605.13087#pg1>.

Meng: For deployment, this means we can potentially use smaller models like the 244M version because the R-MFT recipe makes them match or exceed the performance of those larger 769M counterparts on spontaneous speech <ref:2605.13087#pg1> <ref:2605.13087#pg1>.

Lalam: I think what matters most is that adaptation efficiency is governed by two things: timing of large parameter updates and ordering of acoustic complexity <ref:2605.13087#pg1>.

Tom: Exactly, so we’re seeing a pattern here where the way you introduce that initial high learning rate sets up the whole process for success or failure <ref:2605.13087#pg3>.

Jane: That gives us a clear path forward for anyone working on low-resource languages, showing them exactly what kind of training recipe makes sense to test first <ref:2605.13087#pg1>.