RuleSmith: Multi-Agent LLMs for Automated Game Balancing

arXiv:2602.06232 · cs.LG, cs.AI, cs.GT, cs.MA · Submitted 2026-02-05 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "RuleSmith: Multi-Agent LLMs for Automated Game Balancing".

Jane: Here's the gist of the paper. The gist RuleSmith introduces a framework that achieves automated game balancing by leveraging multi-agent LLMs for self-play and Bayesian optimization over a multi-dimensional rule space.

Jane: RuleSmith couples a game engine, multi-agent LLMs self-play, and Bayesian optimization operating over a multi-dimensional rule space <ref:2602.06232#pg6>.

Paper summary: Tom: The paper introduces RuleSmith as a framework that uses multi-agent LLMs for self-play combined with Bayesian optimization over a parameterized rule space.

Jane: It’s a proof of concept built on CivMini, which is this simplified civilization-style game with two asymmetric factions governed by twelve tunable parameters.

Lu: The agents interpret textual rulebooks and game states to propose actions that are then validated by the game engine, which yields noisy estimates of faction win rates.

Meng: Their primary implementation uses Gaussian-process Bayesian optimization with Expected-Improvement-based adaptive sampling to allocate more evaluation games to promising candidates.

Lalam: They consistently find near-balanced configurations: three independent optimization runs reach near-even empirical win rates in independent two hundred and three hundred game evaluations, and additional experiments with NTBEA and TPE show that the same self-play interface supports different search components.

Tom: They also mention cross-model evaluations and small human studies that provide preliminary evidence of transfer across different player populations.

Jane: What's interesting is that RuleSmith produces interpretable numerical rule adjustments without needing task-specific policy training from the LLMs themselves.

Lu: It’s treating the game as a parameterized asymmetric environment and directly optimizing the rule space using multi-agent LLM playtests as evaluations, which differs from procedural content generation methods like those mentioned in other work.

Meng: They focus on abstract, simplified environments for this design-time tool rather than real-time decision execution, which is important for deployment concerns.

Lalam: The limitations they flag are that the LLM agents are imperfect proxies for human players or domain experts, and their experiments are conducted in a simplified environment with fully observable state and relatively small action spaces.

Conclusion: Tom: So to wrap up, this paper on "RuleSmith: Multi-Agent LLMs for Automated Game Balancing" shows how you can use multi-agent LLMs to automate game balancing by setting up self-play against a Bayesian optimizer over a rule space.

Jane: It’s about using those agents to generate actions based on the rules and then feeding the results into an optimization loop that tries to find settings where both factions have near equal win rates.

Lu: What this means is that we can move away from relying entirely on expert intuition for balancing complex, rule-driven systems and instead use these LLMs as flexible evaluators.

Meng: It’s a design-time tool, focused on finding the right parameters without needing to train the agents for a specific game strategy.

Lalam: They found that under different agent configurations with comparable capacity, the optimized parameters stay largely symmetric, which suggests it works across different setups.

Tom: The authors are Ziyao Zeng, Chen Liu, Tianyu Liu, Hao Wang, Xiatao Sun, Fengyu Yang, Xiaofeng Liu and Zhiwen Fan.

Jane: This work points toward a way to analyze complex rule-driven environments by treating the game as a parameterized system you can optimize for directly.

Lu: It suggests that LLMs are capable of interpreting structured rules well enough to serve as powerful simulators in this context.

Meng: The potential application here is not just games, but maybe policy design or resource allocation where interactions are rule-based and asymmetric.

Lalam: RuleSmith is designed as an offline analysis tool, which gives us a scalable way to explore the configuration space for things that are otherwise too costly to analyze manually.

Ziyao Zeng, Chen Liu, Tianyu Liu, Hao Wang, Xiatao Sun, Fengyu Yang, Xiaofeng Liu, Zhiwen Fan

Yale University

cs.LG, cs.AI, cs.GT, cs.MA

Submitted: 2026-02-05

Updated: 2026-10-21

Comments: Accepted at NeurIPS 2026

Project page: https://adonis-galaxy.github.io/RuleSmith-website

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 83/100

The gist: The gist RuleSmith introduces a framework that achieves automated game balancing by leveraging multi-agent LLMs for self-play and Bayesian optimization over a multi-dimensional rule space.

Key concepts

RuleSmith: A framework that combines multi-agent LLMs for self-play with Bayesian optimization over a parameterized rule space. It treats LLMs as flexible evaluators to automatically discover game rules that achieve desired balance.
CivMini: A simplified, turn-based strategy game used as a proof of concept. It features two asymmetric factions—Empire and Nomads—with distinct unit capabilities and victory conditions, allowing the framework to test rule adjustments.
Bayesian Optimization with Adaptive Sampling: The optimization method used to search for good rules. It uses a surrogate model to suggest promising rule changes and then adaptively decides how many evaluation games are needed for those candidates, focusing computational effort where it matters most.
Multi-agent LLM Self-Play: Two LLM agents interpret the game's text rules and current state to propose actions simultaneously. The game engine validates these actions, creating a feedback loop where the agents learn and refine their strategies through repeated interaction.

Terminology

Bayesian Optimization — 2× in this paper · explained in 3 episodes

Summary

The gist

RuleSmith introduces a framework that achieves automated game balancing by leveraging multi-agent LLMs for self-play and Bayesian optimization over a multi-dimensional rule space.

How it works

RuleSmith couples a game engine, multi-agent LLMs self-play, and Bayesian optimization operating over a multi-dimensional rule space <ref:2602.06232#pg6>.
It instantiates two LLM agents that read the rulebook and generate actions for all units to play the game <ref:2602.06232#pg6>.
The environment is implemented as a Python game engine, which at each decision point constructs a structured natural language description of the current game state <ref:2602.06232#pg6>.
The state description is concatenated with a role-specific system prompt (for Empire vs. Nomads) and retrieved rules from the RAG system <ref:2602.06232#pg6>.
Since we use multi-action mode, the agent is asked to produce a JSON object containing actions for all units simultaneously <ref:2602.06232#pg6>.
The game engine parses the response, validate that each action is legal for its unit, and either execute it or fall back to a safe default (e.g., PASS) if parsing fails or an illegal action is proposed <ref:2602.06232#pg6>.

The Optimization Pipeline

The goal is to choose θ such that both roles are equally strong in expectation, encoded as minimizing a balance loss L(θ) = wE − 0.5 + wN − 0.5 + 0.5 · wD <ref:2602.06232#pg6>.
A Bayesian optimizer maintains a surrogate model g(θ) over a continuous relaxation of the rule space and selects new candidates ˜θt+1 by maximizing an acquisition function <ref:2602.06232#pg6>.
The number of games Nt is adaptively determined based on the Expected Improvement (EI) of the proposed candidate, where promising candidates receive more games for accurate evaluation <ref:2602.06232#pg6>.
The adaptive sampling strategy dynamically allocates more evaluation games to promising candidates (those with high Expected Improvement), concentrating computational resources where they matter most <ref:2602.06232#pg6>.
Each continuous proposal is mapped to a valid, discrete ruleset via a deterministic discretization operator D(·) before evaluation <ref:2602.06232#pg6>.

The Game Environment (CivMini)

CivMini is a minimalistic two-faction, turnbased asymmetric strategy game inspired by 4X games such as Civilization <ref:2602.06232#pg6>.
There are two asymmetric factions: Empire and Nomads <ref:2602.06232#pg6>.
Each faction starts with a city tile (Empire at position (1, 1), Nomads at (5, 5)) and a small number of units placed at fixed coordinates <ref:2602.06232#pg6>.
Empire fields a specialized economy: its Farmer units can only gather resources (but cannot battle), and its Soldier units can only engage in combat (but cannot gather) <ref:2602.06232#pg6>.
Nomads deploy a single versatile Cavalry unit type that can battle with higher mobility than Empire units <ref:2602.06232#pg6>.
Victory and scoring are determined by the final score for each faction, which is a weighted combination of (i) remaining resources, (ii) the number of battles won, and (iii) the number of surviving units <ref:2602.06232#pg6>.

Empirical Validation and Results

Experiments show that RuleSmith can discover rulesets where the win-rate difference between asymmetric factions drops to 0%, even from intentionally imbalanced initializations <ref:2602.06232#pg6>.
Through systematic evaluations on CivMini across different model sizes (2B, 8B) and faction configurations, we demonstrate that our RuleSmith, combining Bayesian optimization with acquisition-based adaptive sampling, consistently achieves near-balanced outcomes (50% ± 5% win rates) <ref:2602.06232#pg6>.
Table 4 shows that Bayesian Optimization with adaptive sampling consistently converges to a near-equal win rate <ref:2602.06232#pg6>.
RuleSmith can obtain balanced game parameters under different game designs with varying map sizes and game turns <ref:2602.06232#pg6>.
Across different LLM agent configurations, the optimized parameters remain largely symmetric when both agents have comparable capacity, leading to balanced time-to-kill (TTK) values in both directions <ref:2602.06232#pg6>.
Under asymmetric pairings, the optimization jointly adjusts economy, combat, production, and scoring parameters to compensate for differences in agent capability <ref:2602.06232#pg6>.

Limitations

The LLM agents used in RuleSmith are imperfect proxies for human players or domain experts <ref:2602.06232#pg6>.
Our experiments are conducted in a simplified, abstract game environment with fully observable state and relatively small action spaces <ref:2602.06232#pg6>.
While RuleSmith can discover multiple balanced parameterizations for a given agent pairing, it does not provide formal guarantees of global optimality or robustness under distribution shifts <ref:2602.06232#pg6>.

This work advances automated design tools and multi-agent simulation by enabling LLM-based evaluation and optimization of rule-driven systems <ref:2602.06232#pg6>. By treating large language models as flexible evaluators rather than deployed decisionmakers, RuleSmith offers a scalable and interpretable approach to analyzing complex, parameterized environments <ref:2602.06232#pg6>. Potential applications of this framework extend beyond games to domains such as policy design, economics, security, and medical decision-making where rule-based asymmetric interactions are the norm <ref:2602.06232#pg6>. RuleSmith is designed as an offline analysis and design-time tool operating on abstract, simplified environments rather than a system for real-time decision execution <ref:2602.06232#pg6>. By treating large language models as flexible evaluators rather than deployed decisionmakers, RuleSmith offers a scalable and interpretable approach to analyzing complex, parameterized environments <ref:2602.06232#pg6>. Potential applications of this framework extend beyond games to domains such as economic modeling, policy analysis, and resource allocation where outcomes depend on structured rules and multi-actor interactions <ref:2602.06232#pg6>. RuleSmith is designed as an offline analysis and design-time tool operating on abstract, simplified environments rather than a system for real-time decision execution <ref:2602.06232#pg6>. By treating large language models as flexible evaluators rather than deployed decisionmakers, RuleSmith offers a scalable and interpretable approach to analyzing complex, parameterized environments <ref:2602.06232#pg6>. Potential applications of this framework extend beyond games to domains such as economic modeling, policy analysis, and resource allocation where outcomes depend on structured rules and multi-actor interactions <ref:2602.06232#pg6>. RuleSmith is designed as an offline analysis and design-time tool operating on abstract, simplified environments rather than a system for real-time decision execution <ref:2602.06232#pg6>. By treating large language models as flexible evaluators rather than deployed decisionmakers, RuleSmith offers a scalable and interpretable approach to analyzing complex, parameterized environments <ref:2602.06232#pg6>. Potential applications of this framework extend beyond games to domains such as economic modeling, policy analysis, and resource allocation where outcomes depend on structured rules and multi-actor interactions <ref:2602.06232#pg6>. RuleSmith is designed as an offline analysis and design-time tool operating on abstract, simplified environments rather than a system for real-time decision execution <ref:2602.06232#pg6>. By treating large language models as flexible evaluators rather than deployed decisionmakers, RuleSmith offers a scalable and interpretable approach to analyzing complex, parameterized environments <ref:2602.06232#pg6>. Potential applications of this framework extend beyond games to domains such as economic modeling, policy analysis, and resource allocation where outcomes depend on structured rules and multi-actor interactions <ref:2602.06232#pg6>. RuleSmith is designed as an offline analysis and design-time tool operating on abstract, simplified environments rather than a system for real-time decision execution <ref:2602.06232#pg6>. By treating large language models as flexible evaluators rather than deployed decisionmakers, RuleSmith offers a scalable and interpretable approach to analyzing complex, parameterized environments <ref:2602.

Improvements for AI systems

Automated Game Balancing using LLM Self-Play for Rule Optimization: The improved AI system can automatically discover rule parameters that achieve near-equal win rates between asymmetric factions by leveraging multi-agent LLMs to play self-play games and Bayesian optimization to search the rule space.
Zero-Shot Rulebook Interpretation and Action Generation: The system can interpret textual rulebooks and game states to generate legal actions for all units in a parameterized game, as demonstrated by the capability of LLM agents read the rulebook and generate actions for all units to play the game.
Efficient Parameter Space Search via Adaptive Sampling: The AI can efficiently search high-dimensional rule spaces by integrating Bayesian optimization with acquisition-based adaptive sampling, which dynamically allocates more evaluation games to promising candidates (those with high Expected Improvement), concentrating computational resources where they matter most.
Interpretable Rule Adjustment Insights: The system provides interpretable insights into game fairness by showing how health scaling, resource efficiency, and production tempo jointly determine fairness in competitive systems, allowing for direct application of adjusted parameters to downstream game systems.

Abstract

Game balancing is a longstanding challenge requiring repeated playtesting, expert intuition, and extensive manual tuning. We introduce RuleSmith, a framework that couples multi-agent LLM self-play with surrogate-based optimization over a parameterized rule space. As a proof of concept, we build CivMini, a simplified civilization-style game with two asymmetric factions governed by 12 tunable parameters. LLM agents interpret textual rulebooks and game states to propose actions that are validated by the game engine, yielding noisy estimates of faction win rates. Our primary implementation uses Gaussian-process Bayesian optimization with Expected-Improvement-based adaptive sampling to allocate more evaluation games to promising candidates. RuleSmith consistently finds near-balanced configurations: three independent optimization runs reach near-even empirical win rates in independent 200- and 300-game evaluations, and additional experiments with NTBEA and TPE show that the same self-play interface supports different search components. Cross-model evaluations and small human studies provide preliminary evidence of transfer across player populations. RuleSmith produces interpretable numerical rule adjustments without task-specific policy training. Project and code: https://adonis-galaxy.github.io/RuleSmith-website/