Can LLMs Access their World Knowledge for Event Prediction?
cs.CL
Submitted: 2026-10-13
Updated: 2026-10-13
Comments: Accepted to Findings of EMNLP 2026
Code: https://github.com/BorealisAI/DescribeEvent
License: http://creativecommons.org/licenses/by-nc-sa/4.0/
The gist: Recent work has questioned whether large language models (LLMs) can be useful for temporal event prediction.
Abstract
Recent work has questioned whether large language models (LLMs) can be useful for temporal event prediction. Despite vast pretrained world knowledge, LLMs often underperform simpler specialized models. We ask whether this knowledge is truly inaccessible, or whether current methods fail to present event prediction in a form LLMs can use. We argue for the latter and show that LLMs become effective event forecasters when the interface is designed around two choices. The first grounds each event in a natural-language description, and the second separates event semantics from the prediction space to prevent semantic bias from label associations. To test this, we introduce DescribeEvent, a benchmark of 150K events across seven datasets and six domains with event-level descriptions. We find that capable LLMs substantially benefit from descriptions and symbolic labels, outperforming description-free variants and non-LLM baselines, while weaker models gain little. These results suggest that accessing LLM world knowledge for event prediction is an interface problem, and that event descriptions offer a practical way to elicit the knowledge LLMs already encode.
Sources
- A Comprehensive Evaluation of Large Language Models on Temporal Event Forecasting
- Enhancing Temporal Awareness in LLMs for Temporal Point Processes
- Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities
- The Llama 3 Herd of Models
- Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders
- GPT-4o System Card
- TPP-LLM: Modeling Temporal Point Processes by Efficiently Fine-Tuning Large Language Models
- Tokenization counts: the impact of tokenization on arithmetic in frontier LLMs
- Inferring Events from Time Series using Language Models
- Reducing Distraction in Long-Context Language Models by Focused Learning
- Qwen3 Technical Report
- MIRAI: Evaluating LLM Agents for Event Forecasting
Related papers
- Exploring Solution Divergence and Its Effect on Large Language Model Problem Solving
- Ishigaki-IDS-Bench: A Benchmark for Generating Information Delivery Specification from BIM Information Requirements
- Subliminal Steering: Stronger Encoding of Hidden Signals
- MedStruct-S: A Benchmark for Key Discovery, Key-Conditioned QA and Semi-Structured Extraction from OCR Clinical Reports
- The End of Transformers? On Challenging Attention and the Rise of Sub-Quadratic Architectures
- Untangling the Mechanisms of Misleading Context in Medical Question Answering