Language Models as AI Research World Models
cs.CL
Submitted: 2026-10-08
Updated: 2026-10-08
Code: https://github.com/allenai/OLMo-core
License: http://creativecommons.org/licenses/by/4.0/
The gist: AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement.
Terminology
Abstract
AI research agents automate the cycle of proposing, implementing, and evaluating experiments, opening a path toward recursive self-improvement. Yet their ability to propose experiments outpaces their capacity to execute them in real environments, making outcome prediction a key capability for sustained self-improvement under limited experimental budgets. We investigate language models as Research World Models (RWMs), which predict the outcomes of candidate interventions across research environments. Our evaluation draws on over 2,600 experimental records from nine research environments spanning pretraining, post-training, and inference, representing more than 171,000 H100 GPU-hours of experimentation. Research knowledge acquired from real experimental experience improves RWM predictions of unseen interventions within the same environment (Spearman +0.27), and can be reused across environments. For example, using only pretraining experience from OLMo3, Marin, and Nanochat, an RWM reduces selection regret in the Qwen3 environment by 78% compared with zero-experience setting. These benefits extend to multi-round Autoresearch under a fixed selection budget: RWMs with in-env and cross-env research knowledge increase the best gain achieved by 15.8% and 11.6%, respectively. Ablations across 13 language models used as RWMs show that adding research knowledge can improve intervention ranking more than changing models or increasing reasoning effort alone. These findings support language models as RWMs and motivate accumulating experimental data for future RWM training.
Sources
- Encoder-Decoder Diffusion Language Models for Efficient Training and Inference
- ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models
- Accelerating Neural Architecture Search using Performance Prediction
- MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering
- Evaluating Large Language Models Trained on Code
- Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models
- AI Research Preference Models
- What Can Transformers Learn In-Context? A Case Study of Simple Function Classes
- AutoLLMResearch: Training Research Agents for Automating LLM Experiment Configuration - Learning from Cheap, Optimizing Expensive
- World Models
- Dream to Control: Learning Behaviors by Latent Imagination
- Learning Latent Dynamics for Planning from Pixels
- Mastering Diverse Domains through World Models
- AIRA_2: Overcoming Bottlenecks in AI Research Agents
- Training Compute-Optimal Large Language Models
- MLAgentBench: Evaluating Language Agents on Machine Learning Experimentation
- When to Trust Your Model: Model-Based Policy Optimization
- Scaling Laws for Neural Language Models
- In-context Reinforcement Learning with Algorithm Distillation
- AI-Supervisor: Autonomous AI Research Supervision via a Persistent Research World Model
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering