ContextWeaver: Selective and Dependency-Structured Memory Construction for LLM Agents

The gist The ContextWeaver framework introduces a selective and dependency-structured memory system for LLM agents that organizes interaction history into a graph of reasoning steps to select relevant context for future actions.

ContextWeaver Framework

ContextWeaver is a selective and dependency-structured memory framework that organizes an agent’s interaction trace into a graph of reasoning steps and selects the relevant context for future actions It consists of three main components: (1) a dependency-based construction module that links each reasoning step to the earlier steps it relies on; (2) compact dependency summaries that condense root-to-step reasoning paths into reusable units; and (3) a lightweight validation layer that incorporates execution feedback Unlike prior context management approaches, ContextWeaver supports: (1) dependency-based construction and traversal that link each step to the earlier steps it relies on; (2) compact dependency summarization that condenses root-to-step reasoning paths into reusable units; and (3) a lightweight validation layer that incorporates execution feedback These components enable ContextWeaver to preserve the context most relevant to the agent’s next action while remaining within token budgets.

Dependency-Aware Context Construction

The core of ContextWeaver is dependency-aware context construction, which maintains a dependency graph based on logical and causal relationships and weaves context for the next decision of the LLM agent. This process involves several steps outlined in Algorithm 1: (1) Node Extraction, where the latest history entry is converted into a structured node Nk; (2) Parent Selection, where candidate parents Sk are identified by an LLM-based Logical Dependency Analyzer based on estimated likelihood pi that Nk depends on Ni; (3) Ancestry Dependency Construction, which retrieves the dependency subgraph from the identified ancestry set A via a breadth-first traversal upward through the dependency graph; and (4) Context Weaving, where context C is constructed according to A by retaining full T–A–O triples for nodes within this subgraph and compressing Observation Oi for all other history entries.

Dependency Summarizer

For interpretability, each node maintains a brief dependency summary that records the main reasoning dependency from the root to that node. ContextWeaver constructs this summary incrementally by providing the dependency summaries of parents and the current node’s summary, rather than re-summarizing the full history at every step. This incremental approach offers two advantages: (1) it substantially reduces computational cost by avoiding repeated processing of the entire history, and (2) it preserves a compact, dependency-structure memory of how earlier steps guide later reasoning.

Validation and Test Layer

To incorporate execution feedback, ContextWeaver adds a validation layer that records outcomes from each testing or verification step. This layer provides two complementary signals: Fine-grained evidence, where a TestTracker parses tool observations to extract individual test results into the field test results; and Node-level abstraction, where the ValidationDetector assigns each node a simple label, validation status ∈ ⊆ passed, failed, unknown or superseded. Nodes marked as failed are skipped when searching for parent candidates, ensuring new steps never depend on incomplete or broken states.

Experimental Results

Experiments on SWE-BENCH Verified and Lite benchmarks show that ContextWeaver improves performance over a sliding-window baseline in pass@1 while reducing reasoning steps and token usage. In the Unified setting, Claude Sonnet 4 benefits from ContextWeaver, reaching 66.0% on the Verified split compared to 63.2% under the sliding-window baseline. The framework is particularly effective when early structural signals are informative, as seen in the django django-14631 case where ContextWeaver consistently succeeds by preserving long-range dependencies between early architectural analysis and later implementation steps.

Conclusion

ContextWeaver organizes an agent’s history into a dependency-structured memory that retains only the steps needed to support future actions. The results indicate that maintaining explicit dependencies can improve tool use reliability under a fixed context budget. Future work includes improving parent selection with advanced testing signals and developing smaller controller models to reduce overhead.

Ethics Statement

All experiments use publicly available benchmarks and open-source repositories with no human subjects involved

Limitations

Our work focuses on structured, test-driven agent environments such as SWE-BENCH, where explicit validation signals are available. Extending ContextWeaver to handle more diverse reasoning signals is a promising direction for future work. The behavior of the controller ultimately depends on the reliability of the underlying language model.

References

Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI conference on artificial intelligence

Alexis Chevalier, Alexander Wettig, Anirudh Ajith, and Danqi Chen. Adapting language models to compress contexts

Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, Carlos E. Jimenez, John Yang

Luyu Gao, Aman Madaan, Shuyan Zhou

Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Llmlingua: Compressing prompts for accelerated inference of large language models

Carlos E Jimenez, John Yang, Alexander Wettig

Minki Kang, Wei-Ning Chen, Dongge Han, Huseyin A Inan, Lukas Wutschitz

Tianle Li, Ge Zhang, Quy Duc Do, Xiang Yue

Yucheng Li, Bo Dong, Frank Guerin, and Chenghua Lin. Compressing context to enhance inference efficiency of large language models

Tobias Lindenbauer, Igor Slinko, Ludwig Felder

Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape

Tianyang Liu, Canwen Xu, and Julian McAuley.

1. Bold header: Enhanced Context Management for Tool-Using Agents

Improvement: Implement ContextWeaver to replace simple sliding window mechanisms by organizing interaction history into a graph of reasoning steps. This allows the system to select context based on causal and logical structure needed for multi-step reasoning rather than just recency, which prevents the agent from breaking ongoing plans.

2. Bold header: Dependency-Aware Context Construction

Improvement: Integrate the dependency-aware context construction module where each reasoning step is converted into a structured node that stores its state and identifies the earlier steps it relies on. This ensures that the final context passed to the LLM agent is a selective, dependency-aware memory state that preserves key evidence for the next action under strict token constraints.

3. Bold header: Compact Dependency Summarization

Improvement: Utilize the Dependency Summarizer to produce compact dependency summaries by incrementally building them from parent summaries and the current node's summary. This provides a mechanism to condense root-to-step reasoning paths into reusable units while preserving the structural relationships, avoiding flat textual condensations that discard the structural relationships among reasoning steps.

4. Bold header: Execution Feedback Integration

Improvement: Incorporate a lightweight validation layer that records execution outcomes and annotates nodes with a reliability signal. This allows the system to filter out unreliable nodes, ensuring that new steps never depend on incomplete or broken states by skipping candidates marked as failed during parent selection.

5. Bold header: Improved Reasoning Stability

Improvement: Leverage the dependency graph structure to improve run-to-run stability, as demonstrated by ContextWeaver achieving a lower standard deviation in performance compared to the sliding-window baseline on 100 instances. This suggests that maintaining explicit dependencies can improve tool use reliability under a fixed context budget and mitigate stochastic generation noise.

6. Bold header: Strategic Context Selection based on Task Type

Improvement: The system will dynamically adjust memory selection based on task complexity, as shown in the case study where dependency-aware memory is most beneficial when tasks require maintaining relationships across distant steps or files. This allows the agent to favor structural context for complex, multi-file coordination tasks while defaulting to recency-based selection for strictly linear debugging trajectories.

Large language model (LLM) agents often struggle in long-context interactions. As the history grows, sliding windows and prompt compression may drop earlier information that later steps rely on. Retrieval-based memory systems surface relevant content but still overlook the causal and logical dependencies between steps. We introduce ContextWeaver, a selective and dependency-structured memory framework that organizes an agent's interaction trace into a graph of reasoning steps and selects the relevant context for future actions. ContextWeaver supports (1) dependency-based construction and traversal that link each step to the earlier steps it relies on; (2) compact dependency summaries of the reasoning paths; and (3) a lightweight validation layer that incorporates execution feedback. On five software-engineering and terminal benchmarks with a fixed budget per task, ContextWeaver improves the resolve rate over a sliding-window baseline on all five, with a significant gain when pooled across benchmarks, while using 13-18% fewer agent steps. It also performs favorably against alternative context-management approaches, with the largest gains on long-horizon repository tasks. On SWE-Bench Pro, ContextWeaver raises the resolve rate from 66.5% to 72.6%, with significant improvements across all three model backbones. The improvement also holds when the sliding window's budget is raised to cover the summary cost.