Memorizing Who, Knowing How: Benchmarking Personal and Procedural Memory for Persistent Assistants
cs.AI
Submitted: 2026-10-14
Updated: 2026-10-20
License: http://creativecommons.org/licenses/by/4.0/
The gist: Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act.
Terminology
Abstract
Memory systems for persistent personal agents must move beyond isolated memories, maintaining both personal memory about whom they serve and procedural memory about how to act. Crucially, in harness-based personal assistance, procedures or skills are inherently user-conditioned: personal context determines which procedures apply, how they are instantiated, and when they should change. Existing benchmarks typically emphasize either personal or procedural memory, leaving their interaction underexplored. We introduce PraxisMemBench, a harness-based benchmark for persistent personal assistance across daily-life and work trajectories, where personal and procedural memory evolve together and later tasks depend on both. Experiments show that maintaining two memory types separately is insufficient, with completion dropping from single-type to interdependent dependencies (84.6% to 77.9%). More broadly, across existing methods, relation-related failures increase from early to late sessions as interaction history grows (59.8% to 72.9%), revealing a continual-maintenance challenge: writing correct memories is not enough if their evolving content and organization cannot be preserved for later retrieval. To address this, we propose PraxisMem, which uses privileged joint-state supervision to internalize the ability to maintain memory content and organization into the memory manager, with location-aware GRPO and dynamic residual-routed OPSD. PraxisMem reaches 40.78% overall Session Success Rate, compared with 20.60% for Claude Code's native memory.
Sources
- Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction
- Scaling Synthetic Data Creation with 1,000,000,000 Personas
- VitaBench 2.0: Evaluating Personalized and Proactive Agents in Long-Term User Interactions
- CL-bench Life: Can Language Models Learn from Real-Life Context?
- MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks
- ClawArena: Benchmarking AI Agents in Evolving Information Environments
- PersonaMem-v2: Towards Personalized Intelligence via Learning Implicit User Personas and Agentic Memory
- SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks
- MemOS: A Memory OS for AI System
- MemGPT: Towards LLMs as Operating Systems
- Zep: A Temporal Knowledge Graph Architecture for Agent Memory
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models
- Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents
- DeltaMem: Towards Agentic Memory Management via Reinforcement Learning
- Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models
- SemaClaw: A Step Towards General-Purpose Personal AI Agents through Harness Engineering
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems