MMLA-RSI: Recursive Self-Improvement through RTT and RTMA
cs.CL, cs.LG
Submitted: 2026-10-14
Updated: 2026-10-14
Comments: Technical report. 24 pages, 5 figures, 12 tables
License: http://creativecommons.org/licenses/by/4.0/
The gist: An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains.
Abstract
An agent can improve its behavior within an unresolved problem by changing how it acts and which experience it retains. We propose MMLA-RSI, a recursive self-improvement scheme with two implementations in Memory-Mediated Learning Architecture: Reasoning-Time Training (RTT) changes the policy state Φ, while Reasoning-Time Memory Adaptation (RTMA) changes authoritative memory M. Each route also makes its adaptation method an object of improvement. A current numerical updater trains the policy proposing its successor; a current memory manager chooses the records available to the process proposing its successor. An independently admitted successor takes over the same role in the next generation. We give bounded executable rule languages, shared task/proposal interfaces, and one algorithm for either route or their composition. The RTT instance changes scalar schedules in a clipped-momentum update. The RTMA instance chooses NULL or a complete-row write with possible eviction. A fixed interpreter, trusted committer, and independent evaluator bound what can change. State versions, read closures, and conditional properties describe causal execution, accepted transitions, and finite work. Existing numerical-component studies motivate the construction. In four-candidate program selection, retaining LARC feedback updates reduces expected error by 24.65 and 36.65 percentage points for two trained initializations. In chronological continuous-integration replay, feedback binding is positive while retention increases loss. Completed SQL studies improve substantially over an unadapted substrate but do not establish the required additional online benefit. These results characterize their declared components. The two recursive routes and their joint realization are specified here; their beneficial recursive chains and discovery efficiency remain to be evaluated.
Sources
- MMLA: Memory-Mediated Learning Architecture for Predictive Dual-State Adaptation
- LARC: Low-Rank Adaptive Residual Connections for Learning in Frozen Models
- On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes
- Learning to learn by gradient descent by gradient descent
- Symbolic Discovery of Optimization Algorithms
- Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution
- Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
- LoRA: Low-Rank Adaptation of Large Language Models
- Eliminating Meta Optimization Through Self-Referential Meta Learning
- Agentic-TTT: Training test-time policy for test-time training
- Self-Refine: Iterative Refinement with Self-Feedback
- On First-Order Meta-Learning Algorithms
- Goedel Machines: Self-Referential Universal Problem Solvers Making Provably Optimal Self-Improvements
- Reflexion: Language Agents with Verbal Reinforcement Learning
- Learning to (Learn at Test Time): RNNs with Expressive Hidden States
- Test-Time Training with Self-Supervision for Generalization under Distribution Shifts
- No Time Like the Present: Agentic Test-Time Training for LLM Agents
- RRSI: Regularized Recursive Self-Improvement of Agent Harnesses
- Recursive Self-Improvement via On-Policy Distillation for Reasoning
- CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering