Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Clarify or Answer".

Jane: Here's the gist of the paper. The gist: CoA, an ask-or-answer agentic framework, separates when to ask for clarification from when to answer directly in context-dependent Visual Question Answering tasks.

Tom: CoA Framework CoA is proposed as an agentic framework that explicitly separates the decision of whether to answer from clarification generation, consisting of three components: (i) a controller that decides whether the agent should answer or ask for clarification, (ii) a clarification policy that generates a single targeted clarification question when needed, and (iii) an answering model that produces the final answer either directly or after incorporating the clarification response CoA consists of three components: (i) a controller that decides whether the agent should answer or ask for clarification, (ii) a clarification policy that generates a single targeted clarification question when needed, and (iii) an answering model that produces the final answer either directly or after incorporating the clarification response CoA executes a single-step clarification interaction: it either answers immediately or asks exactly one clarification question and then answers.

Jane: Controller Learning Objective The controller is trained to predict whether missing context is required to answer reliably, which is formulated as binary classification over u ∈ 2 where u ∈ ANSWER, CLARIFY.

Tom: The objective for the controller is optimized using the standard cross-entropy loss Lctrl = − log πctrl(u∗ I, q), where u∗is the ground-truth ask-or-answer decision.

Title and authors: Tom: So let's talk about who wrote this. The paper is titled Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification, and the authors are Zongwan Cao, Bingbing Wen, and Lucy Lu Wang from the University of Washington.

Jane: It’s a team that seems to have a deep background in both visual question answering and reinforcement learning. They aren't just looking at how to make models answer questions; they are focused on the specific problem where the image alone isn't enough context >

Lu: I think their work is interesting because they’re not just tweaking an existing model. They are designing a whole framework, CoA, specifically for this context-dependent ambiguity that comes from combining an image and a question together.

Meng: What does that mean practically? Does it give us something we can deploy right now, or is this theoretical stuff? Because I need to know what the actual cost of running this kind of agent would be.

Tom: It’s designed to be an agentic framework, meaning it has a defined flow. It uses a controller, a clarification policy, and an answering model. This structure is what makes it interesting from an engineering standpoint because you have clear roles for each part >

Jane: So when we look at the title again, "Clarify or Answer," they are addressing that core dilemma in VQA—when is the risk of being wrong too high to just guess? It sets up a very clear decision point for the AI.

Lalam: It’s about making that decision process explicit so it doesn't just make a blind prediction when it can be sure >

The paper's summary: Tom: So the summary of Clarify or Answer explains that real-world visual question answering is often context-dependent. An image and a question might both look fine on their own, but together they might lead to multiple valid answers depending on something outside the picture >

Jane: They point out that in previous work, ambiguity was usually in the question itself or the image content separately. This paper focuses specifically on that combination—the interaction between them—to find where context is missing >

Lu: The core contribution is CoA, which explicitly models this separation. It first uses a controller to decide if clarification is necessary. If it asks for one specific question, the answering model then produces the final answer, either directly or after using that clarification response >

Meng: So they’re essentially building a safety switch for when the AI hits a wall of uncertainty and needs to pause and seek external help before committing to an output >

Tom: Right. And they introduce CONTEXTCLARIFY, which is their dataset of ambiguous VQA pairs, along with a contrast set that shows which ones are actually clear and which ones require clarification >

Jane: The summary emphasizes that the method is built around reinforcement learning, specifically GRPO-CR, to optimize those generated questions. They are using rewards to make sure the questions they generate are good—well-formed, relevant, and actually helpful in resolving the ambiguity >

Lalam: So it’s not just about asking *any* question; it’s about training the AI to ask the *right* question that actually moves us toward a correct answer >

The paper's improvements: Tom: Now let's look at what they claim as improvements. They say training the controller with supervised fine-tuning substantially improves ambiguity detection over zero-shot prompting on CONTEXTCLARIFY, and that performance gains generalize to the ClearVQA dataset >

Jane: That’s a big step for robustness. It means we can train the AI to be good at spotting when it needs help, and that skill sticks even when we move to a new set of questions it hasn't seen before >

Lu: They also highlight that GRPO-CR consistently improves clarification question quality over both prompting and supervised fine-tuning, and this quality improvement holds up even on out-of-distribution datasets >

Meng: I’m interested in the generalization part. If the model learns how to ask better questions on a training set, but those questions don't work when the input distribution shifts, that's not useful for real deployment >

Tom: But they argue that because of their reward design—things like format compliance and relevance—the policy learns a more robust way to generate good questions, which is what helps it generalize >

Jane: The paper also shows that end-to-end VQA accuracy improves by an average of plus fifteen point three points, which is reported across three different VLLMs and three different datasets >

Lalam: That fifteen point three point gain across those models suggests the framework isn't just a small tweak; it genuinely helps the final answer quality when you use this CoA setup >

Conclusion: Tom: So to wrap up, Clarify or Answer presents an agentic framework that clearly separates the decision of whether to ask or answer from what question to ask if clarification is needed. It shows that reinforcement learning with GRPO-CR produces clarification questions that are more effective at resolving ambiguity than just prompting or supervised fine-tuning >

Jane: The implication for us is that explicit clarification has real value in context-dependent visual reasoning, and this method consistently improves both the quality of the questions asked and the final answer accuracy across different AI models >

Lu: It’s a solid piece of research because it tackles ambiguity from a specific angle—the interaction between image and question—and shows how an agentic loop can be structured to handle that uncertainty >

Meng: I still have my reservations about deployment. The paper mentions the limitation that it focuses on single-factor underspecification, meaning if your real-world problem has multiple sources of missing context, this framework might not scale easily >

Lalam: That’s true. And they flag that the method needs joint optimization of all three components—the controller, the policy, and the answering model—which is a lot to manage in practice >

Tom: Exactly. They also mention ethical considerations because if deployed without safeguards, models could ask inappropriate or biased questions if they aren't careful about how they are implemented >

Jane: So while this framework shows strong performance gains on these specific tasks, the authors themselves point out that it doesn't cover every possible kind of real-world ambiguity yet >