When Agent Memory Needs to Compute: Evidence Access and Executable Aggregation
cs.AI
Submitted: 2026-10-14
Updated: 2026-10-17
License: http://creativecommons.org/licenses/by/4.0/
The gist: Questions about an agent's history often require aggregating events, which retrieval alone does not resolve.
Terminology
Abstract
Questions about an agent's history often require aggregating events, which retrieval alone does not resolve. We combine source-preserving evidence access with executable aggregation. Trajectory Memory (TM) retains the original trajectory and constructs a source-addressable reading view at question time. TallyMem augments it with a reusable event ledger, model-generated event specifications and deterministic execution. With a commit-specific adaptation on 776 questions from 99 held-out MINTEval repositories, TallyMem raises Qwen3-32B exact match from 35.2% to 49.0%, a 13.8-point gain over TM; gains span 12.8-13.9 points across three readers. A controlled probe with event sets parsed independently from raw logs shows an 11.0-point gain, from 82.8% to 93.8%, without routing. On full AMA-Bench, the routed system yields a smaller 1.3-point gain, from 69.8% to 71.1%. These results support executable aggregation on tasks that require combining events, with benefits dependent on the task and event specification, the largest residual system-error category in our diagnostic analysis.
Sources
- gpt-oss-120b & gpt-oss-20b Model Card
- Language Models Enable Simple Systems for Generating Structured Views of Heterogeneous Data Lakes
- Oolong: Evaluating Long Context Reasoning and Aggregation Capabilities
- Universal Self-Consistency for Large Language Model Generation
- Binding Language Models in Symbolic Languages
- Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory
- Language models fail at extended rule following
- MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
- RULER: What's the Real Context Size of Your Long-Context Language Models?
- ChatDB: Augmenting LLMs with Databases as Their Symbolic Memory
- MINTEval: Evaluating Memory under Multi-Target Interference in Long-Horizon Agent Systems
- MemTrace: Probing What Final Accuracy Misses in Long-Term Memory
- MemGPT: Towards LLMs as Operating Systems
- Executable Code Actions Elicit Better LLM Agents
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory
- MemGym: a Long-Horizon Memory Environment for LLM Agents
- Qwen3 Technical Report
- Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQL
- ReAct: Synergizing Reasoning and Acting in Language Models
- AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems