Game Arena: Strategic LLM Evaluation in Competitive Environments
cs.AI
Submitted: 2026-09-25
Updated: 2026-09-25
Comments: 31 pages, 15 figures. Technical report. Project page: https://www.kaggle.com/game-arena
Code: https://github.com/google-deepmind/game_arena
License: http://creativecommons.org/licenses/by/4.0/
The gist: We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games.
Terminology
Abstract
We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation. This technical report details the infrastructure behind Game Arena and describes the three pilot game environments: Chess, Poker, and Werewolf. These environments span perfect information, imperfect information, and multiplayer game settings, enabling a systematic study of models' strategic planning, adaptation, and robustness under uncertainty. For each game, we provide a detailed description of the environment, evaluation metrics, and results from running full competitions across models. Through robust infrastructure and large-scale ground-truth based evaluation, Game Arena ensures reproducibility, transparency and generalizability to new games and variants over time.
Sources
- Training Verifiers to Solve Math Word Problems
- LiveBench: A Challenging, Contamination-Limited LLM Benchmark
- AI Agents That Matter
- AgentBench: Evaluating LLMs as Agents
- GTBench: Uncovering the Strategic Reasoning Limitations of LLMs via Game-Theoretic Evaluations
- PokerBench: Training Large Language Models to become Professional Poker Players
- Game Reasoning Arena: A Framework and Benchmark for Assessing Reasoning Capabilities of Large Language Models via Game Play
- ReAct: Synergizing Reasoning and Acting in Language Models
- OpenSpiel: A Framework for Reinforcement Learning in Games
- Evaluating Agents using Social Choice Theory
- DSGBench: A Diverse Strategic Game Benchmark for Evaluating LLM-based Agents in Complex Decision-Making Environments
Related papers
- MAVEN-T: Reinforced Heterogeneous Distillation for Real-Time Multi-Agent Trajectory Prediction
- Model Discovery Agent: LLM-assisted Bayesian experiment design for data-efficient discovery of mechanistic world models
- Agents' Last Exam
- The Clinician's Veto: Navigating Trust, Liability, and Uncertainty in Autonomous AI Prescribing
- MindHelper: Closed-Loop Embodied Mental-State Reasoning for Precision Intervention
- Incumbent Advantage: Brand Bias and Cognitive Manipulation Dynamics in LLM Recommendation Systems