Large Language Models over Networks: Collaborative Intelligence under Resource Constraints

arXiv:2605.08626 · eess.SP, cs.DC, cs.LG, cs.MA · Submitted 2026-05-09 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Hi, I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "Large Language Models over Networks".

Jane: Here's the gist of the paper. The gist Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving, but they require collaborative intelligence to serve diverse applications under heterogeneous resource constraints.

Tom: Collaborative Intelligence Paradigm The article focuses on "collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages".

Jane: This collaboration strives for "superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers".

Tom: The collaboration is presented along two complementary dimensions: "vertical device-cloud collaboration and horizontal multi-agent collaboration".

Paper summary: Tom: So we’ve seen how these different models can team up across devices and clouds to handle tasks, which is what this paper is about here, "Large Language Models over Networks: Collaborative Intelligence under Resource Constraints."

Jane: Right, so essentially this paper looks at how multiple independent LLMs can collaborate—through natural language or structured messages—when you have all these tricky resource limits like computation, memory, and communication bandwidth.

Lu: The authors are focusing on collaboration as a way to get better response quality when you’re juggling those different tiers of network access.

Meng: From my side, I’m thinking about what this means for deployment at scale; if we can get this kind of resource balancing working without needing a massive infrastructure, that changes how we build things.

Lalam: I think the core idea is that you don't need one perfect AI endpoint; you need a system that knows which AI is best for each part of the job at any moment.

Tom: Exactly, so it’s about moving away from relying on just one big model and instead building this intelligent distribution of work across the network.

Jane: It suggests that for things like autonomous driving or smart assistants, we can achieve a better balance between speed, quality, and cost than if we just picked the single most capable AI available.

Lu: The paper explores how you can learn these collaboration policies—how to decide which model goes where—by optimizing for things like quality versus latency and cost over an entire conversation.

Meng: And that’s where I get interested; if the learning mechanism is robust enough, it could mean we don't have to manually tune every single endpoint decision.

Lalam: It really points toward a future where the intelligence isn't just packed into one giant model, but spread out across the entire network structure itself.

Tom: So what this really means for us on the ground is that we start thinking about building these distributed architectures as a standard way to serve AI applications.

Jane: It opens up a whole new area of design where the system is smart enough to know when it needs local power and when it needs cloud muscle, all based on the real-time situation.

Lu: The next thing we’ll look at is how they handle those real-world challenges like scaling under different resource limits and making sure this collaborative intelligence remains trustworthy.

Conclusion: Tom: So we’re wrapping up today on "Large Language Models over Networks: Collaborative Intelligence under Resource Constraints."

Jane: Right, so this paper really centers on how multiple independent LLMs can team up across devices and clouds to handle tasks when you have all these tough resource limits.

Lu: The authors are showing that the way you distribute those LLMs—vertically and horizontally—and then learn the rules for that distribution, actually helps improve response quality when you’re juggling computation, memory, communication, and cost across different network tiers.

Meng: What this means practically is we stop thinking about just one big model endpoint and start building systems where the intelligence is spread out based on what's available right now.

Lalam: It points toward a future where the AI isn't just packed into one giant model, but in this distributed way that allows models to intelligently hand off and share knowledge as the user moves between a phone and the cloud.

Tom: Exactly, so we’re moving past picking one model blindly and instead building these intelligent distribution systems that balance quality against latency and cost in real time.

Jane: And for applications like autonomous driving or smart assistants, this suggests we can get better service by letting different AI models talk to each other intelligently instead of forcing one model to do everything or relying on just one endpoint.

Lu: The big shift is moving from monolithic solutions toward these distributed architectures where the intelligence is spread across the network infrastructure itself rather than concentrated in a single large model instance.

Meng: It means that for my team, the focus has to shift toward building those robust communication protocols and policies that manage this collaboration efficiently so we don't run out of bandwidth or get hit with unexpected cost spikes during high-volume inference.

Lalam: And they’re essentially saying that the way these LLMs interact at the task level, guided by learned policies, is what unlocks this ability to balance all those competing resource demands simultaneously.

Tom: So ultimately, it’s about making sure that when you’re dealing with intermittent connectivity or tight latency budgets on a phone, the system can intelligently decide whether to use a small local model or offload to a powerful cloud model based on what the current conversation needs.

Jane: That dynamic decision-making across devices and clouds is what makes this paper so relevant for applications that are moving beyond simple chatbots into real-time, constrained environments where every millisecond and every byte counts.

Lu: The implication is that we need to focus on how these learned policies scale when you introduce new LLMs and new devices into the ecosystem, which will be a big part of the next research challenge.

Liangqi Yuan, Wenzhi Fang, Shiqiang Wang, H. Vincent Poor, Christopher G. Brinton

Purdue University · University of Exeter · Princeton University

eess.SP, cs.DC, cs.LG, cs.MA

Submitted: 2026-05-09

Updated: 2026-10-18

Comments: IEEE Communications Magazine

License: http://creativecommons.org/licenses/by/4.0/

The gist: The gist Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving, but they require collaborative intelligence to serve diverse

Key concepts

Collaborative Intelligence: A system where several independent LLMs distributed across devices and the cloud communicate via natural language or structured messages to solve a task. The goal is to get better results by leveraging the strengths of different models while managing constraints like speed, memory, and cost.
Vertical Device-Cloud Collaboration: This strategy involves an on-device LLM handing off complex parts of a task to a more powerful cloud LLM. It requires intelligent selection mechanisms to decide which model handles which part and effective methods for managing the flow of information between the two endpoints.
Horizontal Multi-Agent Collaboration: This involves multiple peer LLMs working together as agents, such as debating or dividing labor to solve a problem collectively. The design focuses on how these agents interact—whether they debate or work in parallel—and how their communication structure impacts overall performance and efficiency.

Terminology

Resource Constraints — 2× in this paper · explained in 5 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 8 episodes
Multi-agent Collaboration — 1× in this paper · explained in 2 episodes

Summary

The gist Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving, but they require collaborative intelligence to serve diverse applications under heterogeneous resource constraints.

Collaborative Intelligence Paradigm

The article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. This collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. The collaboration is presented along two complementary dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration.

Collaborative Inference Architectures

Collaborative inference can be organized into two main topologies:

Vertical Device-Cloud Collaboration: This involves on-device LLMs offload difficult tasks upward to more capable cloud LLMs. A key aspect of this is the Joint LLM and Modality Selection, where the system must decide which LLM should serve it. Furthermore, Context Management across Turns addresses the overhead of transferring accumulated dialogue history when switching endpoints.
Horizontal Multi-Agent Collaboration: This occurs where peer LLM agents collaborate to solve tasks collectively, and the collaboration patterns can be characterized by interaction pattern (e.g., debate, division-of-labor) and execution structure (e.g., parallel, sequential). Communication topology is also a critical design choice that affects performance.

Learning to Collaborate

The paper examines how these collaboration strategies are learned through two learning objectives:

Routing Policy Learning: In device-cloud collaboration, this involves training policies that learn to make this choice by balancing quality, latency, and cost. This can be achieved via a Router-Based Selection, where a classifier predicts the best candidate based on offline profiling. For multi-turn settings, the routing problem is modeled as a Markov decision process to optimize the cumulative quality-cost tradeoff over an entire conversation.
Cooperative Capability Learning: This addresses how LLMs learn to work together effectively in multi-agent settings through Cooperative Policy Optimization, where multiple LLMs are trained in authentic collaborative scenarios. Additionally, Inter-Agent Network Optimization tackles communication efficiency by replacing fully connected graphs with sparse topologies to reduce token costs while encouraging independent reasoning.

Case Study and Challenges

A case study on device-cloud routing in multi-modal conversations shows that a learned policy breaks out of this tradeoff curve, achieving comparable response quality at substantially lower latency and cost. Open challenges include Scaling Under Resource Heterogeneity, which requires building protocols that remain effective as new LLMs and devices enter the ecosystem, and ensuring Trustworthy Collaborative Intelligence to manage privacy risks and defend against cascading attacks.

Collaborative intelligence will become a foundational design principle for networked AI systems in the years ahead.

Improvements for AI systems

The system can achieve a quality-latency-cost balance beyond what any individual endpoint offers by jointly navigating multiple competing objectives under fluctuating conditions, as demonstrated in Fig 2 of the paper. This allows applications to select endpoints based on real-time resource constraints like battery life, link reliability, and per-token pricing.
The system can implement a learned routing policy that breaks out of the quality-latency cost tradeoff curve of prompt-based approaches by internalizing cumulative resource consumption as part of its state rather than deciding per query. This makes routing decisions sequential and stateful across multi-turn conversations, optimizing the cumulative quality-cost tradeoff over an entire session.
The system can function in multi-modal conversational scenarios by jointly optimizing LLM selection and input composition to substantially reduce communication cost while preserving response quality. This involves deciding which subset of inputs to transmit based on which LLM will process them, thereby balancing marginal information gain against transmission overhead.
The system can support horizontal multi-agent collaboration where multiple independent LLM instances cooperate through message passing, such as debate or division-of-labor patterns, regardless of the agents' physical location. This allows tasks to decompose into parallel subtasks assigned to agents best equipped for them, optimizing execution structure and communication topology.
The system can dynamically adapt its collaboration strategy by training routing policies using reinforcement learning where the state encodes the current query, conversation history, and resource usage so far. This enables the system to reason about when to spend resources over a conversation rather than whether to spend them on average.
The system can improve communication efficiency in multi-agent settings by replacing fully connected graphs with sparse communication topologies for peer-level message exchange. This reduces token costs while matching or exceeding performance by limiting each agent's view and encouraging more independent reasoning.
The system can prevent the propagation of harmful instructions across a distributed workflow by developing defenses that detect and contain cascading effects in real time within multi-tier collaborative systems. This addresses the risk of prompt injection persisting covertly within accumulated context, especially when LLM agents control physical systems like vehicles or robots.

Abstract

Large language models (LLMs) are transforming society, powering applications from smartphone assistants to autonomous driving. Yet cloud-based LLM services alone cannot serve a growing class of applications, including those operating under intermittent connectivity, sub-second latency budgets, data-residency constraints, or sustained high-volume inference. On-device deployment is in turn constrained by limited computation and memory. No single endpoint can deliver high-quality service across this spectrum. This article focuses on collaborative intelligence, a paradigm in which multiple independent LLMs distributed across device and cloud endpoints collaborate at the task level through natural language or structured messages. Such collaboration strives for superior response quality under heterogeneous resource constraints spanning computation, memory, communication, and cost across network tiers. We present collaborative inference along two complementary and composable dimensions: vertical device-cloud collaboration and horizontal multi-agent collaboration, which can be combined into hybrid topologies in practice. We then examine learning to collaborate, addressing the training of routing policies and the development of cooperative capabilities among LLMs. Finally, we identify open research challenges including scaling under resource heterogeneity and trustworthy collaborative intelligence.