Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification

arXiv:2601.16400 · cs.CL · Submitted 2026-01-23 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Clarify or Answer".

Jane: Here's the gist of the paper. The gist: CoA, an ask-or-answer agentic framework, separates when to ask for clarification from when to answer directly in context-dependent Visual Question Answering tasks.

Tom: CoA Framework CoA is proposed as an agentic framework that explicitly separates the decision of whether to answer from clarification generation, consisting of three components: (i) a controller that decides whether the agent should answer or ask for clarification, (ii) a clarification policy that generates a single targeted clarification question when needed, and (iii) an answering model that produces the final answer either directly or after incorporating the clarification response CoA consists of three components: (i) a controller that decides whether the agent should answer or ask for clarification, (ii) a clarification policy that generates a single targeted clarification question when needed, and (iii) an answering model that produces the final answer either directly or after incorporating the clarification response CoA executes a single-step clarification interaction: it either answers immediately or asks exactly one clarification question and then answers.

Jane: Controller Learning Objective The controller is trained to predict whether missing context is required to answer reliably, which is formulated as binary classification over u ∈ 2 where u ∈ ANSWER, CLARIFY.

Tom: The objective for the controller is optimized using the standard cross-entropy loss Lctrl = − log πctrl(u∗ I, q), where u∗is the ground-truth ask-or-answer decision.

Title and authors: Tom: So let's talk about who wrote this. The paper is titled Clarify or Answer: Reinforcement Learning for Agentic VQA with Context Under-specification, and the authors are Zongwan Cao, Bingbing Wen, and Lucy Lu Wang from the University of Washington.

Jane: It’s a team that seems to have a deep background in both visual question answering and reinforcement learning. They aren't just looking at how to make models answer questions; they are focused on the specific problem where the image alone isn't enough context >

Lu: I think their work is interesting because they’re not just tweaking an existing model. They are designing a whole framework, CoA, specifically for this context-dependent ambiguity that comes from combining an image and a question together.

Meng: What does that mean practically? Does it give us something we can deploy right now, or is this theoretical stuff? Because I need to know what the actual cost of running this kind of agent would be.

Tom: It’s designed to be an agentic framework, meaning it has a defined flow. It uses a controller, a clarification policy, and an answering model. This structure is what makes it interesting from an engineering standpoint because you have clear roles for each part >

Jane: So when we look at the title again, "Clarify or Answer," they are addressing that core dilemma in VQA—when is the risk of being wrong too high to just guess? It sets up a very clear decision point for the AI.

Lalam: It’s about making that decision process explicit so it doesn't just make a blind prediction when it can be sure >

The paper's summary: Tom: So the summary of Clarify or Answer explains that real-world visual question answering is often context-dependent. An image and a question might both look fine on their own, but together they might lead to multiple valid answers depending on something outside the picture >

Jane: They point out that in previous work, ambiguity was usually in the question itself or the image content separately. This paper focuses specifically on that combination—the interaction between them—to find where context is missing >

Lu: The core contribution is CoA, which explicitly models this separation. It first uses a controller to decide if clarification is necessary. If it asks for one specific question, the answering model then produces the final answer, either directly or after using that clarification response >

Meng: So they’re essentially building a safety switch for when the AI hits a wall of uncertainty and needs to pause and seek external help before committing to an output >

Tom: Right. And they introduce CONTEXTCLARIFY, which is their dataset of ambiguous VQA pairs, along with a contrast set that shows which ones are actually clear and which ones require clarification >

Jane: The summary emphasizes that the method is built around reinforcement learning, specifically GRPO-CR, to optimize those generated questions. They are using rewards to make sure the questions they generate are good—well-formed, relevant, and actually helpful in resolving the ambiguity >

Lalam: So it’s not just about asking *any* question; it’s about training the AI to ask the *right* question that actually moves us toward a correct answer >

The paper's improvements: Tom: Now let's look at what they claim as improvements. They say training the controller with supervised fine-tuning substantially improves ambiguity detection over zero-shot prompting on CONTEXTCLARIFY, and that performance gains generalize to the ClearVQA dataset >

Jane: That’s a big step for robustness. It means we can train the AI to be good at spotting when it needs help, and that skill sticks even when we move to a new set of questions it hasn't seen before >

Lu: They also highlight that GRPO-CR consistently improves clarification question quality over both prompting and supervised fine-tuning, and this quality improvement holds up even on out-of-distribution datasets >

Meng: I’m interested in the generalization part. If the model learns how to ask better questions on a training set, but those questions don't work when the input distribution shifts, that's not useful for real deployment >

Tom: But they argue that because of their reward design—things like format compliance and relevance—the policy learns a more robust way to generate good questions, which is what helps it generalize >

Jane: The paper also shows that end-to-end VQA accuracy improves by an average of plus fifteen point three points, which is reported across three different VLLMs and three different datasets >

Lalam: That fifteen point three point gain across those models suggests the framework isn't just a small tweak; it genuinely helps the final answer quality when you use this CoA setup >

Conclusion: Tom: So to wrap up, Clarify or Answer presents an agentic framework that clearly separates the decision of whether to ask or answer from what question to ask if clarification is needed. It shows that reinforcement learning with GRPO-CR produces clarification questions that are more effective at resolving ambiguity than just prompting or supervised fine-tuning >

Jane: The implication for us is that explicit clarification has real value in context-dependent visual reasoning, and this method consistently improves both the quality of the questions asked and the final answer accuracy across different AI models >

Lu: It’s a solid piece of research because it tackles ambiguity from a specific angle—the interaction between image and question—and shows how an agentic loop can be structured to handle that uncertainty >

Meng: I still have my reservations about deployment. The paper mentions the limitation that it focuses on single-factor underspecification, meaning if your real-world problem has multiple sources of missing context, this framework might not scale easily >

Lalam: That’s true. And they flag that the method needs joint optimization of all three components—the controller, the policy, and the answering model—which is a lot to manage in practice >

Tom: Exactly. They also mention ethical considerations because if deployed without safeguards, models could ask inappropriate or biased questions if they aren't careful about how they are implemented >

Jane: So while this framework shows strong performance gains on these specific tasks, the authors themselves point out that it doesn't cover every possible kind of real-world ambiguity yet >

Zongwan Cao, Bingbing Wen, Lucy Lu Wang

University of Washington · Allen Institute for AI

cs.CL

Submitted: 2026-01-23

Updated: 2026-10-20

Code: https://github.com/zwverse/clarifyor-answer

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: The gist: CoA, an ask-or-answer agentic framework, separates when to ask for clarification from when to answer directly in context-dependent Visual Question Answering tasks.

Key concepts

CoA Framework: A three-part agentic system: a controller that decides to answer or ask for clarification, a policy that generates one targeted question if needed, and an answering model. It executes a single-step interaction: either answer immediately or ask exactly one question before answering.
Controller Learning Objective: The controller is trained via binary classification to predict whether missing context is required to answer reliably (ANSWER or CLARIFY). This training uses cross-entropy loss, teaching the agent when it should seek external information instead of guessing.
GRPO-CR: A reinforcement learning method used to train the clarification policy. It samples candidate questions and receives a reward based on factors like question format, relevance to missing context, and whether answering would change the final outcome. This guides the model to generate high-quality clarifying questions.
ContextCLARIFY Dataset: A curated dataset of 275 ambiguous image-question pairs derived from CODIS. It includes both cases where clarification is needed and where it is not, helping train the system to distinguish between solvable and unsolvable visual ambiguities.

Terminology

Reinforcement Learning — 2× in this paper · explained in 4 episodes
Accuracy — 1× in this paper · explained in 1 episode
GRPO — 1× in this paper · explained in 6 episodes

Summary

The gist: CoA, an ask-or-answer agentic framework, separates when to ask for clarification from when to answer directly in context-dependent Visual Question Answering tasks.

CoA Framework

CoA is proposed as an agentic framework that explicitly separates the decision of whether to answer from clarification generation, consisting of three components: (i) a controller that decides whether the agent should answer or ask for clarification, (ii) a clarification policy that generates a single targeted clarification question when needed, and (iii) an answering model that produces the final answer either directly or after incorporating the clarification response CoA consists of three components: (i) a controller that decides whether the agent should answer or ask for clarification, (ii) a clarification policy that generates a single targeted clarification question when needed, and (iii) an answering model that produces the final answer either directly or after incorporating the clarification response CoA executes a single-step clarification interaction: it either answers immediately or asks exactly one clarification question and then answers.

Controller Learning Objective

The controller is trained to predict whether missing context is required to answer reliably, which is formulated as binary classification over u ∈ 2 where u ∈ ANSWER, CLARIFY. The objective for the controller is optimized using the standard cross-entropy loss Lctrl = − log πctrl(u∗ I, q), where u∗is the ground-truth ask-or-answer decision.

Clarification Policy Learning

For ambiguous inputs, the clarification policy is trained using reinforcement learning with GRPO-CR (Clarification Reasoning), which is a reinforcement learning approach built on Group Relative Policy Optimization (GRPO). The policy samples a group of N candidate clarifications and receives a scalar reward R(q(i)c), and the loss is computed as LGRPO-CR = −Ei log πclr(q(i)c I, q) · Aˆi. The reward design includes several components: a Format reward (γformat), which encourages outputs that are well-formed, concise clarification questions; a Focused relevance reward (γrelevance), which encourages questions that explicitly target the missing context; an Ambiguity resolution reward (γresolution), which rewards questions whose answers would alter the final VQA outcome; a Novelty reward (γnovelty) to penalize trivial rephrasings of the original question; and a Ground-truth check reward (γgroundtruth) measuring alignment with human-authored reference clarifications.

Dataset and Training

CONTEXTCLARIFY is a curated dataset of 275 ambiguous image-question pairs derived from CODIS, supplemented with manually sourced examples annotated with a human-verified open-ended clarification question. The contrast set is constructed by augmenting each ambiguous instance with context-completed and irrelevant-context variants, yielding both cases where clarification is necessary and cases where it is not. The dataset splits into training, validation, and test sets following a 70/15/15 ratio.

Results

CoA achieves consistent improvements at both the module and system levels, improving end-to-end VQA accuracy by an average of +15.3 points (83%) over prompting-based baselines across three VLLMs and three datasets. Controller performance shows that training the CoA controller with SFT substantially improves ambiguity detection on CONTEXTCLARIFY over zero-shot prompting and performance improvements generalize to ClearVQA. GRPO-CR consistently improves clarification question quality over both prompting and supervised fine-tuning, generalizing to OOD datasets. End-to-end VQA accuracy shows that CoA consistently outperforms zero-shot prompting for all evaluated models on CONTEXTCLARIFY and generalizes to the OOD ClearVQA dataset. Human evaluations confirm that GRPO-CR consistently achieves the highest human ambiguity resolution scores across model variants, outperforming both prompting and supervised fine-tuning.

Discussion

The work highlights the value of explicit clarification for context-dependent visual and textual reasoning. CoA executes a single-step interaction, which is attractive when interaction is costly, but future work could incorporate cost-sensitive decision rules so the controller can adapt its behavior to deployment constraints.

Conclusion

CoA presents an ask-or-answer agentic framework that separates when to ask (controller) from what to ask (clarification policy), and produces a final answer either directly or after incorporating a single clarification response. It demonstrates that reinforcement learning with GRPO-CR produces clarification questions that are more effective at resolving ambiguity than direct prompting or supervised baselines. Across module- and systemlevel evaluations, and different model families, CoA consistently improves clarification quality and end-to-end answer accuracy compared to vanilla prompting and supervised baselines.

Limitations

Limitations include broader coverage and generalization, as CONTEXTCLARIFY focuses on single-factor underspecification, and the need for joint optimization of all components rather than treating the answering model as a fixed backbone. Ethical considerations mention that models may ask inappropriate or biased questions if deployed without safeguards. Potential risks include models asking sensitive information if deployed without safeguards.

Use of AI Assistants

AI assistants were used to support paper writing, develop data-processing and evaluation scripts, simulate clarification responses, compute similarity-based rewards, and generate initial drafts of clarification questions that are subsequently reviewed and/or modified by the authors.

References

The references listed in the paper include Agrawal et al. 2016 on Vqa. Luo et al. 2024 on context-dependent ambiguity. Jian et al. 2025 on interactive formulations like ClearVQA. Shao et al. 2024 on GRPO. Rao and Daumé 2018 on learning to ask good questions. Chenhang Cui et al. 2023 on visual ambiguity. Yue Fan et al. 2025 on Grit. Fuwen Luo et al. 2024 on Codis. Qwen 7B/3B models technical reports from 2024. Rafael Rafailov et al. 2023 on DPO. Bingbing Wen et al. 2024 on Characterizing LLM abstention behavior. Yiwei Yang et al. 2025 on Escaping the spuriverse. Jihan Yao et al. 2025 on Mmmg. Michael Zhang and Eunsol Choi 2023 on Clarify when necessary. Sudha Rao and Hal Daumé 2018 on Learning to ask good questions. Meet Shah et al. 2019 on Cycle-consistency for robust visual question answering. Zhihong Shao et al. 2024 on Deepseekmath. Xiaolong Wang et al.

Improvements for AI systems

Improve Ambiguity Detection via Controller Training on Mixed Data: Training the CoA controller with SFT consistently improves ambiguity detection over zeroshot prompting on CONTEXTCLARIFY and generalizes to the ClearVQA setting. This allows the system to reliably distinguish between questions that can be answered directly and those requiring external context, even when encountering novel visual or linguistic patterns.
Enhance Clarification Question Generation with GRPO-CR: GRPO-CR consistently improves clarification question quality over both prompting and supervised fine-tuning, and generalizes to OOD datasets. This ensures that the generated questions are not just syntactically correct but are explicitly optimized for resolving ambiguity through a tailored reward structure that captures format compliance, focused relevance to missing context, and ambiguity resolution potential.
Implement Context-Aware Decision Making: The system can now execute a single-step clarification interaction: it either answers immediately or asks exactly one clarification question and then answers. This constraint prevents over-asking while enabling the recovery of missing contextual factors in underspecified cases, as detailed in Section 3.5.
Strengthen Robustness to Distribution Shift: The end-to-end performance demonstrates that CoA improves end-toend VQA accuracy by selectively asking clarifying questions, and these gains transfer to the OOD ClearVQA dataset. This indicates the framework is not overfit to CONTEXTCLARIFY but can successfully adapt its clarification strategy when applied to different visual question distributions.
Enable Multi-Turn Interaction Planning: A future extension could be a multi-turn variant of CoA that can ask a small number of clarification questions with an explicit stopping rule, balancing answer reliability against interaction cost. This would allow the agent to manage costly interactions by defining termination criteria based on uncertainty reduction and potential gain.