Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models

arXiv:2605.15706 · cs.LG · Submitted 2026-05-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models".

Jane: Here's the gist of the paper. The gist The Differentiable Mixture-of-Agents (DMoA) is a self-evolving multi-agent framework that enables elastic and adaptive agent collaboration during inference, allowing it to dynamically route and activate agents at each reasoning step to implicitly simulate diverse communication topologies and adapt to evolving demands DMoA Framework DMoA is introduced as a self-evolving multi-agent system that adaptively routes agents in each reasoning step based on the contextual information, thus adapting to the varying demands during execution.

Tom: It extends the Mixture-of-Agents (MoA) concept by allowing agents to be spontaneously determined in each reasoning step, which can directly simulate workflows of many multi-agent systems like Chain and Star.

Jane: This approach resolves the static compilation dilemma in graph-based multi-agent systems by dynamically adjusting the communication topology along the temporal dimension.

Tom: The core motivation is that multiple agents can mutually benefit from each other’s outputs, and DMoA adopts a structure analogous to Mixture-of-Experts (MoE) where routing appropriate agents effectively and efficiently addresses the current step-wise demand.

Title and authors: Tom: So we're looking at this paper today, "Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models." It sounds like they are trying to fix a problem where these multi-agent systems, which are getting really popular for complex reasoning, usually have to be set up ahead of time.

Jane: Exactly. The title says "Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models," which makes it sound like they're trying to make the collaboration between these agents more flexible and self-managing during the thinking process, not just a fixed sequence.

Lu: Basically, they are moving away from those pre-defined or pre-compiled communication topologies that we usually see in systems like Chain or Star. They want something that can actually change its structure at every single reasoning step based on what the task is demanding right then and there.

Meng: That sounds ambitious. If the system can change its communication graph dynamically, how does it manage the complexity of training a model that learns to decide which agent to use, and how much to use?

Lalam: It's proposing DMoA, which is this self-evolving framework where agents are determined spontaneously at each step, rather than being fixed in advance. This should allow for a much more adaptive way the system handles the varying demands during execution.

Tom: Right, so we’re talking about dynamic routing instead of static workflows. It's about letting the system decide who talks to whom in real time to see if that helps it solve the problem faster or better.

The paper's summary: Jane: So what’s DMoA actually doing here, beyond just changing the names of things? The summary suggests they design a differentiable, context-aware routing mechanism using recurrent structures to pull in historical and contextual information to make these sparse agent activations.

Lu: They use a Sentence Transformer to compress all that contextual information into one fixed-size semantic vector, which then goes through a recurrent neural network to build cross-layer communications and support autoregressive extension during inference. That’s the core mechanism for how it gets the context.

Tom: I like that part about using a recurrent structure to consider historical routing rationales; that means past decisions influence future choices, which adds a layer of memory to the process. But how do they keep it all differentiable?

Jane: They tackle that by making the routing decision itself differentiable, and they use an adaptive selection method based on equation three—it determines the number of routed agents ki based on whether certain logits are greater than one over N.

Meng: So, not just picking top agents, but dynamically deciding how many to pick based on a threshold derived from those logits? That sounds like it adds a lot of control over resource allocation during inference.

Lalam: It’s about being able to simulate arbitrary communication topologies implicitly because the routing mechanism is designed this way. They are aiming for that "unlimited agents and implicit topologies" idea, which contrasts with the fixed numbers in systems like MoA.

The paper's improvements: Tom: The authors suggest several improvements over existing multi-agent systems, focusing on robustness and ensembling capability as key benefits of DMoA’s elasticity when facing versatile scenarios.

Jane: They show that DMoA is not just about flexibility; it also has better robustness and ensembling capabilities which makes it perform better across different kinds of tasks, even when the situation is unpredictable.

Lu: They are trying to solve the design-then-compile problem by moving toward a self-evolving system where the topology is adaptively designed based on the task, so specialists don't have to fully understand everything beforehand.

Meng: If it’s self-evolving, that means we don't have to spend all our time upfront designing every possible workflow. But what about the cost of training this routing mechanism itself? Is it adding a huge overhead?

Lalam: They optimize this routing mechanism using something called predictive entropy as a self-supervised signal during optimization, which helps guide the system toward better routing decisions without needing external annotations.

Tom: So, they use predictive entropy to create confidence targets and then use a pair-wise ranking loss to align the ordering of those logits with those confidence targets. That’s a clever way to train the selection process.

Conclusion: Jane: So, wrapping up this look at "Differentiable Mixture-of-Agents Incentivizes Swarm Intelligence of Large Language Models," the main point is that DMoA provides an elastic and adaptive agent collaboration during inference that lets it dynamically route and activate agents at each reasoning step.

Tom: It resolves the static compilation dilemma in graph-based multi-agent systems by letting the system implicitly simulate diverse communication topologies, which gives it spatio-temporal flexibility to adapt to evolving demands.

Lu: Ultimately, they’ve designed a differentiable, context-aware routing mechanism that leverages recurrent structures to incorporate historical and contextual information, producing sparse agent activations in a way that encourages swarm intelligence.

Meng: From an engineering standpoint, the system seems promising for deployment because it allows for sparse activation during inference, meaning you can control token consumption by using a summarizer agent to decide when to stop reasoning early.

Lalam: I think this approach is significant because it moves us closer to lifelong swarm intelligence by allowing the agents themselves to learn and evolve their collaboration structure over time.

Xingjian Wu, Junkai Lu, Siyu Yan, Xiangfei Qiu, Jilin Hu, Chenjuan Guo, Bin YangB

East China Normal University

cs.LG

Submitted: 2026-05-15

Updated: 2026-10-17

Code: https://github.com/Significant-Gravitas/AutoGPT

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 89/100

The gist: The gist The Differentiable Mixture-of-Agents (DMoA) is a self-evolving multi-agent framework that enables elastic and adaptive agent collaboration during inference, allowing it to dynamically route

Key concepts

DMoA Framework: A self-evolving multi-agent system that adaptively routes agents in every reasoning step based on context. It extends Mixture-of-Agents by allowing agents to be spontaneously determined, simulating various communication structures like Chain or Star workflows.
Differentiable Routing Mechanism: The core method uses a Sentence Transformer to compress context into a semantic vector and an RNN to consider historical routing decisions. This creates a differentiable router that determines which agents to activate based on the current reasoning step's needs.
Predictive Entropy: A self-supervised signal used during optimization. It measures the uncertainty of predictions across tokens, allowing DMoA to generate confidence targets for agent selection. This helps train the routing mechanism effectively without needing external annotations.
Sparse Activation: During inference, DMoA can activate only a specific number (ki) of agents at each step instead of all N agents. A summarizer agent decides when to stop reasoning, allowing the system to adapt its computational depth based on task complexity.

Terminology

Differentiable Mixture-of-Agents — 2× in this paper
Adaptability — 1× in this paper · explained in 1 episode
Adaptation — 1× in this paper · explained in 1 episode
Benchmark — 1× in this paper · explained in 2 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
Multi-Agent Framework — 1× in this paper · explained in 3 episodes
Multi-Agent System — 1× in this paper · explained in 7 episodes
Robustness — 1× in this paper · explained in 6 episodes

Summary

The gist The Differentiable Mixture-of-Agents (DMoA) is a self-evolving multi-agent framework that enables elastic and adaptive agent collaboration during inference, allowing it to dynamically route and activate agents at each reasoning step to implicitly simulate diverse communication topologies and adapt to evolving demands

DMoA Framework

DMoA is introduced as a self-evolving multi-agent system that adaptively routes agents in each reasoning step based on the contextual information, thus adapting to the varying demands during execution. It extends the Mixture-of-Agents (MoA) concept by allowing agents to be spontaneously determined in each reasoning step, which can directly simulate workflows of many multi-agent systems like Chain and Star. This approach resolves the static compilation dilemma in graph-based multi-agent systems by dynamically adjusting the communication topology along the temporal dimension. The core motivation is that multiple agents can mutually benefit from each other’s outputs, and DMoA adopts a structure analogous to Mixture-of-Experts (MoE) where routing appropriate agents effectively and efficiently addresses the current step-wise demand.

Differentiable Routing Mechanism

The core challenge lies in designing a differentiable routing mechanism that considers contextual information, historical routing decisions, and how to maintain an optimizable structure with a differentiable router. This mechanism is designed using a lightweight Sentence Transformer to compress contextual information into a fixed-size semantic vector. A recurrent neural network is used to consider the historical routing rationales to construct cross-layer communications and support autoregressive extension during inference. The first round of routing is formulated as X1 = SentenceTransformer(Pusr), (1) followed by h1, o1 = RNN-Router(X1), z1 = Linear(o1), (2). The number of routed agents ki is adaptively determined through Equation (3): k i = Min(K, Count(Softmax(z i/τ) ≥ 1/N)), (3). Agents are then selected based on the top-k1 logits in z1, as shown in Equation (4).

Optimization via Predictive Entropy

To optimize the differentiable routing mechanism, DMoA introduces predictive entropy as self-supervised signals to optimize the routing process. The predictive entropy (PE) is defined as the average conditional entropy over all M tokens, PE = -1/M Sum j=1 X j log PΘ(x j x<j), (13). During optimization, DMoA runs all N agents in the i-th reasoning step to collect the prediction entropy of them: Ei ∈ R N. These entropies are normalized through softmax to create confidence targets Ci = Softmax(-Ei) ∈ R N. A pair-wise ranking loss L(i) rank is then adopted to align the ordering of routing logits Zi with Ci: L(i) rank = Sum a,b I(C(a)>C(b)) log 1 + exp − (Z a - Z b), (14). The total loss Lrank across all reasoning steps is calculated as Lrank = 1/L Sum i=1 L(i) rank, (15).

Inference and Adaptivity

During inference, DMoA supports sparse activation to adaptively route ki agents in the i-th reasoning step. The inference procedure involves computing routing logits zi, determining the number of routed agents ki using Equation (3), selecting the top-ki agents and executing them, and aggregating their responses to obtain Xi+1. The summarizer agent is also introduced to either generate the final response or autonomously determine whether more reasoning is required, allowing DMoA to adaptively allocate reasoning depth according to task complexity.

Experimental Results

Extensive experiments across 9 benchmarks demonstrate that DMoA achieves state-of-the-art performance while exhibiting strong efficiency, robustness, and ensembling capabilities. The results show that DMoA achieves average accuracy improvements (against Vanilla) from 5.75 to 26.97 across 9 benchmarks. DMoA possesses more stable performance on diverse tasks, from simple ones on GSM8k (11.72 ↑) and MMLU (10.88 ↑), to hard ones on DS-1000 (25.94 ↑) and DDXPlus (26.97 ↑). DMoA achieves even better performance than closed-source LLMs like GPT 5.2 and Gemini 3 Pro on most settings.

Conclusion

DMoA is a self-evolving multi-agent system capable of elastically extending the communication topology based on the varying task demands, yielding spatio-temporal flexibility. DMoA’s design and optimization jointly provide a solution for lifelong swarm intelligence.

Limitations and Discussion

One limitation is the scalability of the agent pool, as scaling the pool to a much larger number of agents may introduce additional routing difficulty. Another concern is context-length sensitivity, as a single fixed-length semantic vector may not fully preserve the fine-grained structure of long-context dependencies.

Showcases

Case study on GSM8K shows a trajectory involving Math Analyst, Math Solver, Coder Inspector, and Math Analyst steps. Case study on MultiArith involves steps from Math Analyst to Solver and Inspector. Case study on HumanEval shows a sequence involving Algorithm Designer, Coder Test, Analyst, and Bug Fixer steps. Case study on DS-1000 involves steps from Algorithm Designer to Coder Test and Analyst/Bug Fixer. Case study on MMLU involves steps from Teacher Web Searcher to Inspector.

F Details of Baselines

For fair comparison, all baselines use gpt-oss-120b, the same prompt template as DMoA, and decoding temperature 0.1. For method-specific hyperparameters, we follow the original papers whenever available and tune validation-dependent thresholds under the same training budget of 40–80 queries.

A Dataset Details

The evaluation uses nine benchmarks spanning diverse reasoning settings, including MMLU for broad-domain reasoning, GSM8K, MultiArith, SVAMP, and AQuA for mathematical reasoning. HumanEval and DS-1000 are used for coding and program understanding. HotpotQA is used for multi-hop web-based question answering, and DDXPlus is used for medical diagnosis reasoning.

A.3 Prompt Construction

For each routed agent, the input prompt consists of two major parts: Pi j = (Psys,Plast). The system prompt Psys = (Profile i,j,Pusr) contains the role profile of the routed agent together with the original user query. The contextual prompt is constructed as [Context Prompt] given the responses from previous-step agents.

B Training Details

Algorithm 1 summarizes the optimization-time pipeline of DMoA, which involves computing routing logits zi, executing all N candidate agents to collect predictive entropy vector Ei ∈ R N, converting entropy to confidence targets Ci = Softmax(-Ei), and computing the pair-wise ranking loss L(i) rank<ref:2605.

Improvements for AI systems

DMoA enables self evolving multi agent systems to dynamically route and activate agents at each reasoning step, which resolves the static compilation dilemma in graph-based multi-agent systems. This allows for elastic MAS that can extend with varying demands in processing a query, leading to spatio-temporal flexibility.
The differentiable context aware routing mechanism optimizes agent selection through predictive entropy as a self supervised signal, supporting test time training without external annotations. This means the system can adapt during inference by selecting agents based on strong confidence, which may excel at handling current task demands.
The system can achieve superior performance and robustness across diverse benchmarks because DMoA maintains state-of-the-art performance while exhibiting strong efficiency, robustness, and ensembling capabilities. This is demonstrated by its ability to achieve significant accuracy improvements on harder tasks like DS-1000 and DDXPlus compared to baselines.
DMoA supports a flexible extension of reasoning steps with sparse activation during inference, allowing the system to adaptively route ki agents in the i-th reasoning step. This means it can control token consumption by using a summarizer agent to autonomously determine whether the current reasoning process should terminate earlier.
The system demonstrates scalability by showing that accuracy consistently improves with increases in both the agent pool size N and the maximum routing number K, showing that DMoA can elastically coordinate computational resources and activate specialized agents only when they are truly needed.