REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering

arXiv:2511.09966 · cs.CL · Submitted 2025-11-13 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering".

Jane: Here's the gist of the paper. The gist: REAP proposes Recursive Evaluation and Adaptive Planning to enhance Retrieval-Augmented Generation for multi-hop question answering by maintaining structured sub-tasks and facts through a recursive feedback loop between a Sub-task Planner (SP) and a Fact Extractor (FE).

Tom: REAP Framework Overview REAP reframes the multi-hop reasoning process from a linear pipeline into a dynamic, state-driven loop, designed around the core principle of explicitly decoupling planning from fact-gathering to enhance robustness and interpretability It primarily consists of two synergistic modules operating in a recursive loop: the Sub-task Planner (SP) and the Fact Extractor (FE) The overall workflow proceeds as follows: Given a complex query Q, an initial Decomposer first generates a structured task plan, P0 =

p1, p2,..., pN: Each sub-task pi ∈ P0 is a tuple (idi, qi, depsi), representing its unique identifier, query, and dependencies This plan, along with an initially empty facts list F0 = ∅, defines the initial state of the framework The core of REAP is an iterative process that continues until a termination condition is met At each step t, the SP analyzes the current state (Pt−1, Ft−1) to determine a set of executable next actions, Actionst For each sub-task pi ∈ Actionst, the FE performs retrieval and grounded reasoning to produce a new fact object, fi The facts list is updated: Ft = Ft−1 ∪

f1, f2,...: The SP then receives this feedback to generate the subsequent plan, Pt This recursive loop, where the SP guides the FE and the FE’s findings inform the SP’s next planning cycle, continues until the plan is fully resolved Finally, a Synthesizer generates the conclusive answer A by reasoning over the final, comprehensive facts list Ff inal and the original query Q.

Jane: Sub-task Planner (SP) The SP serves as the strategic core of the REAP framework, designed to overcome the limitations of myopic, step-by-step reasoning that often leads to local impasses or deadlocks By receiving and dynamically maintaining the complete initial task plan P0, the SP retains a global perspective throughout the process A key innovation of the SP is its state-aware, modular design Based on the fulfillment level lt of the new fact ft returned by the FE, the SP dispatches the task to one of two specialized sub-modules 1.

Tom: Plan Updater: This sub-module handles ideal scenarios where reasoning progresses as expected (e.g., lt is DirectAnswer) It performs deterministic and rulebased updates to maintain the task plan Its primary functions are Fact Substitution, which systematically rewrites pending sub-tasks by substituting abstract placeholders with concrete entities derived from newly acquired facts and Plan Forking, which in cases where a sub-task yields a set of multiple valid sub-answers, programmatically duplicates the subsequent dependent sub-tasks into parallel branches 2.

Paper summary: Tom: So, we’re looking at this paper called "REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering". Essentially, they’re tackling the problem that current retrieval augmented generation methods struggle with when you have complex multi-hop questions. The main issue is that these existing methods often lack a global plan, which means they can get stuck in local reasoning impasses.

Jane: That sounds like a big headache for any AI trying to answer tricky questions, right? So what REAP claims is that by explicitly maintaining structured sub-tasks and facts through two modules—a Sub-task Planner and a Fact Extractor—they can overcome those limitations. It’s about giving the system a global perspective to guide the whole reasoning direction.

Lu: What I find interesting is how they propose this recursive loop between these two modules, where the planner guides the extraction, and what REAP achieves is that it dynamically optimizes the task-solving trajectory as it goes. They focus on ensuring that both information retrieval and reasoning are tightly coordinated to build a reliable global context.

Meng: From a deployment standpoint, I’m watching how they handle that coordination. The idea of decoupling planning from fact-gathering sounds robust, but we need to see how stable this structure is when the retrieval returns ambiguous or low-quality documents at scale. What breaks at scale is usually in those recovery mechanisms.

Lalam: From my perspective, the paper focuses on creating a representation of global knowledge that gets incrementally enriched through these steps. The goal isn't just getting one answer, but building a logically coherent knowledge base that improves with every fact extracted.

Tom: Exactly, Lalam. And they introduce this unified task paradigm design which they fine-tune across different tasks to make the planning modules much more effective on complex, data-scarce questions where you don't have tons of specific training data for the reasoning steps themselves.

Jane: So it’s not just a single fix for one problem, but a framework designed to be adaptable. It seems like they are trying to make the system smarter about when to plan differently and when to recover from a bad path in the reasoning process.

Lu: The structure itself is what catches my eye here, especially how the Sub-task Planner handles different fulfillment levels of facts returned by the Fact Extractor. It’s not just one way to proceed; it branches into plan updating or re-planning depending on what the fact extraction actually yields.

Meng: Branching sounds complicated to monitor in production. If we have too many parallel branches, tracking which path is most cost-effective becomes a major operational challenge for you, Tom.

Lalam: But the structure is designed to be interpretable by maintaining that structured sub-task plan throughout the entire process. It makes tracing where a specific fact came from much clearer than just looking at one monolithic output.

Tom: Right, so they’re trading a simple linear pipeline for this dynamic, state-driven loop to improve reliability in these tricky multi-hop scenarios. This is definitely aimed at making RAG more dependable for harder tasks.

Conclusion: Jane: So, wrapping up this look at "REAP: Enhancing RAG with Recursive Evaluation and Adaptive Planning for Multi-Hop Question Answering," the authors are Zhu, Zhou, Hong, Liu, and Wang. It’s a framework that essentially rethinks how we combine retrieval with reasoning for multi-hop tasks.

Tom: What this means in plain terms is that instead of just throwing the retrieved text into one big prompt and hoping for the best, you get a system that actively plans its next move based on what information it has gathered so far. It's about making the AI think recursively about which piece of evidence it needs next to solve the whole question.

Lu: The core contribution here is that they introduce this dual-module system—the Sub-task Planner and the Fact Extractor—to ensure that the reasoning trajectory is guided globally, while simultaneously expanding the factual context with reliable pieces of information at each step.

Meng: For a developer or someone shipping this to production, what I see is an attempt to make the planning component more robust against failures in the retrieval step. If one path fails, there's a designated module that can prune that branch and inject a new, logical sequence of steps instead of just crashing. That’s where the operational value lies for me.

Lalam: The implication for us is that we can expect models to handle more complex dependencies in questions because they aren't relying on a single, potentially flawed chain of thought from start to finish. They are building a more traceable and reliable knowledge base throughout the process.

Jane: It suggests that for multi-hop reasoning, the architecture needs to evolve beyond simple retrieval; it needs active, adaptive planning built into the generation loop itself to handle uncertainty better.

Yijie Zhu, Haojie Zhou, Wanting Hong, Tailin Liu, Ning Wang

School of Artificial Intelligence and Computer Science, Jiangnan University

cs.CL

Submitted: 2025-11-13

Updated: 2026-10-17

Comments: published in AAAI 2026

DOI: 10.1609/aaai.v40i41.40830

Code: https://github.com/Deus-Glen/REAP

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 92/100

The gist: The gist: REAP proposes Recursive Evaluation and Adaptive Planning to enhance Retrieval-Augmented Generation for multi-hop question answering by maintaining structured sub-tasks and facts through a

Key concepts

Sub-task Planner (SP): The SP acts as the strategic core, maintaining a global view of the entire task plan. It receives feedback from the Fact Extractor and decides the next action. It uses specialized sub-modules to either update existing steps or initiate a complete overhaul if information is insufficient, preventing local reasoning errors.
Fact Extractor (FE): The FE is responsible for retrieving relevant documents and extracting high-fidelity, structured facts. It doesn't just find answers; it must articulate its reasoning and cite evidence. Crucially, it conditions its extraction on previously gathered facts to perform more sophisticated analysis of the retrieved information.
Recursive Evaluation: This is the core loop where planning and fact-gathering happen repeatedly. The SP plans a step, the FE gathers facts based on that step, and the results are fed back to refine the plan. This recursive feedback ensures that knowledge accumulation is logically coherent and guides the reasoning toward a final solution.
Multi-task Fine-tuning: This strategy trains planning modules (like the Re-Planner) using a unified loss function across different task types. By training on related planning tasks, it allows the model to learn effective planning skills even when data for specific, complex tasks is scarce.

Terminology

RAG — 3× in this paper · explained in 3 episodes
Adaptive Planning — 2× in this paper
Multi-hop Reasoning — 2× in this paper · explained in 3 episodes
Recursive Evaluation and Adaptive Planning — 2× in this paper
Accuracy — 1× in this paper · explained in 1 episode
Core — 1× in this paper · explained in 2 episodes
Fact Extractor — 1× in this paper
Fine-Tuning — 1× in this paper · explained in 5 episodes
Hallucination — 1× in this paper · explained in 4 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
Planner — 1× in this paper · explained in 3 episodes
Retrieval-Augmented Generation (RAG) — 1× in this paper · explained in 21 episodes
Sub-task Planner — 1× in this paper

Summary

The gist: REAP proposes Recursive Evaluation and Adaptive Planning to enhance Retrieval-Augmented Generation for multi-hop question answering by maintaining structured sub-tasks and facts through a recursive feedback loop between a Sub-task Planner (SP) and a Fact Extractor (FE).

REAP Framework Overview

REAP reframes the multi-hop reasoning process from a linear pipeline into a dynamic, state-driven loop, designed around the core principle of explicitly decoupling planning from fact-gathering to enhance robustness and interpretability It primarily consists of two synergistic modules operating in a recursive loop: the Sub-task Planner (SP) and the Fact Extractor (FE) The overall workflow proceeds as follows: Given a complex query Q, an initial Decomposer first generates a structured task plan, P0 = [p1, p2,..., pN] Each sub-task pi ∈ P0 is a tuple (idi, qi, depsi), representing its unique identifier, query, and dependencies This plan, along with an initially empty facts list F0 = ∅, defines the initial state of the framework The core of REAP is an iterative process that continues until a termination condition is met At each step t, the SP analyzes the current state (Pt−1, Ft−1) to determine a set of executable next actions, Actionst For each sub-task pi ∈ Actionst, the FE performs retrieval and grounded reasoning to produce a new fact object, fi The facts list is updated: Ft = Ft−1 ∪ [f1, f2,...] The SP then receives this feedback to generate the subsequent plan, Pt This recursive loop, where the SP guides the FE and the FE’s findings inform the SP’s next planning cycle, continues until the plan is fully resolved Finally, a Synthesizer generates the conclusive answer A by reasoning over the final, comprehensive facts list Ff inal and the original query Q.

Sub-task Planner (SP)

The SP serves as the strategic core of the REAP framework, designed to overcome the limitations of myopic, step-by-step reasoning that often leads to local impasses or deadlocks By receiving and dynamically maintaining the complete initial task plan P0, the SP retains a global perspective throughout the process A key innovation of the SP is its state-aware, modular design Based on the fulfillment level lt of the new fact ft returned by the FE, the SP dispatches the task to one of two specialized sub-modules 1. Plan Updater: This sub-module handles ideal scenarios where reasoning progresses as expected (e.g., lt is DirectAnswer) It performs deterministic and rulebased updates to maintain the task plan Its primary functions are Fact Substitution, which systematically rewrites pending sub-tasks by substituting abstract placeholders with concrete entities derived from newly acquired facts and Plan Forking, which in cases where a sub-task yields a set of multiple valid sub-answers, programmatically duplicates the subsequent dependent sub-tasks into parallel branches 2. Re-Planner: This sub-module is invoked to handle nonideal scenarios (e.g., lt is PartialClue or Failed) It acts as the critical reasoning and recovery mechanism Its hierarchical responsibilities include Pragmatic Sufficiency Assessment, which evaluates whether a partially fulfilled sub-task is functionally sufficient to satisfy the informational requirements of subsequent reasoning steps and Scoped Plan Repair, which executes a major overhaul by pruning the invalid branch and injecting a new, more logical sequence of sub-tasks to create an alternative solution path if information is insufficient.

Fact Extractor (FE)

The facts list F serves as the evolving knowledge foundation for the entire reasoning process; thus, its reliability and comprehensiveness are paramount The Fact Extractor module is designed to extract high-fidelity, structured facts from retrieved documents To mitigate hallucinations and enhance traceability, it requires the LLM to not only provide a conclusive statement but also articulate its reasoning process and cite direct textual evidence A key function of the FE is its ability to discern not just direct answers but also latent clues that may be crucial for subsequent reasoning steps For a given sub-query qt from a sub-task pt ∈ Actionst, the FE process is defined as follows: First, a retriever function fetches a set of relevant documents Dt from an external corpus C. Next, an LLM Mθ processes these documents to generate a new structured fact object ft Crucially, the model is conditioned not only on the current query and retrieved documents but also on the historical facts Ft−1. This contextual conditioning allows the model to perform more sophisticated reasoning, such as coreference resolution and identifying relationships between new information and previously established facts The fact object ft is a structured tuple designed to capture the richness of the extracted information: ft = (st, et, rt, lt) where st is the core statement, a concise, self-contained factual assertion, et ⊆ Dt is the set of direct textual evidence snippets that ground the statement st, rt is the model’s reasoning process, a chain-of-thought explanation of how st is derived from et, and lt is the fulfillment level, a categorical label that classifies the quality of the extracted fact. The statement st and fulfillment level lt are then used to directly guide the SP module’s next planning cycle.

Multi-task Fine-tuning

To enhance the performance of the planning-related modules within the REAP framework, especially for the Re-Planner, which suffers from data scarcity due to its lower invocation frequency, we devise a multi-task fine-tuning strategy The core insight is that despite their varying difficulty, the Decomposer, Plan Updater, and Re-Planner modules share a significant functional commonality: they all require the model to generate or modify a structured task plan based on existing information We leverage this commonality by consolidating their respective datasets, Ddecomp, Dupdate, Dreplan, for joint training of a single planning model Mϕ The training objective is to minimize a weighted joint loss function Lmulti: min ϕ Lmulti(ϕ) = X task∈T λtaskE(x,y)∼Dtask [Ltask(Mϕ(x), y)] This paradigm enables effective knowledge transfer from data-rich tasks (Decomposer, Plan Updater) to the data-scarce Re-Planner task.

Experiments and Results

We conduct evaluations of our method on four multi-hop datasets: HotpotQA (Yang et al. 2018), 2WikiMultihopQA (Ho et al. 2020), MuSiQue (Trivedi et al. 2022b), and Bamboogle (Press et al.

Improvements for AI systems

Recursive Evaluation and Adaptive Planning (REAP) allows LLMs to maintain structured sub-tasks and facts related to the current task through the Sub-task Planner (SP) and Fact Extractor (FE) modules. This enables a dynamic optimization of the task-solving trajectory by explicitly maintaining a global perspective.
Unified Task Paradigm enables effective multi-task fine-tuning, which significantly enhancing SP’s performance on complex, data-scarce tasks by facilitating knowledge transfer from data-rich routine planning to data-scarce critical replanning.
The Fact Extractor module extracts structured facts with a fulfillment level (lt) that is crucial for the SP's decision-making, allowing the system to transition between Plan Updater and Re-Planner based on whether a fact is a DirectAnswer, a PartialClue, or Failed.
The Re-Planner sub-module can perform complex reasoning by executing a Pragmatic Sufficiency Assessment to determine if a partially fulfilled sub-task is functionally sufficient, preventing inefficient search loops.
The system can dynamically assign simple and complex scenarios to different modules, as shown in Table 3, allowing for flexible configuration of computing resources to achieve a delicate balance between performance and efficiency.