ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge

arXiv:2601.13824 · cs.LG · Submitted 2026-01-20 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge".

Jane: Here's the gist of the paper. The gist The proposed ELSA framework systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks.

Tom: How it works ELSA is a novel framework that systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks.

Jane: It introduces three key innovations.

Tom: First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints using public probe inputs and symmetric Kullback-Leibler (KL) divergence.

Paper summary: Tom: So, looking at "ELSA: Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge", we've seen how they integrate split learning with hierarchical federated learning to tackle resource constraints and data heterogeneity.

Jane: They claim this framework works by using a behavior-aware clustering mechanism based on public probes and symmetric Kullback-Leibler divergence, which is then trusted by prediction consistency scores. That’s a complex way of saying they are smart about grouping devices based on their actual predictive logic.

Lu: The core idea is the resource-aware dynamic model partitioning into three segments—clients, edge servers, and the cloud for adapter aggregation—where the split size changes dynamically based on client capacity H n and bandwidth B n.

Meng: And they manage communication with computational sketches and SS-OP to keep data exchanges small while adding a layer of privacy through perturbation. That’s how they handle the cost and privacy issues simultaneously.

Lalam: I'm fairly certain the convergence analysis holds: under Assumptions one through three the sequence of global adapter parameters theta g satisfies a bound on the optimization error involving one/G G X-one g=zero E

| grad F e(theta g)| two: 4L(F e(theta zero) - F e*) sqrt G + sigma two sqrt local G + sigma two two.

Jane: So, if we distill this down for a listener who isn't reading the math, it means they’ve found a way to train very large language models across many small devices without overwhelming their connections or losing privacy.

Tom: It suggests that for deploying LLMs on the edge, we need these kinds of coordinated strategies where you manage the workload distribution and communication costs explicitly rather than assuming everything works smoothly.

Lu: The authors flag a limitation where increasing local blocks to six causes performance degradation because the feature extractor becomes too personalized to the non-IID data distribution. So they set an upper bound on that at six.

Meng: That constraint is important for deployment because it tells us we can't just keep adding layers indefinitely without hurting accuracy on real-world, messy data.

Lalam: The overall implication of ELSA is that we can achieve strong model performance while maintaining low communication latency and managing privacy risks across heterogeneous edge networks.

Conclusion: Tom: So we’re wrapping up our look at ELSA, which stands for Efficient LLM-Centric Split Aggregation for Privacy-Aware Hierarchical Federated Learning over the Network Edge. It’s about how you train big language models on devices that have limited power and slow connections.

Jane: Right, so the authors are showing how you can split a massive model training job up across clients, edge servers, and the cloud without losing data privacy or slowing things down too much.

Lu: Exactly. They use this split learning thing combined with hierarchical federated learning to coordinate all that computation and aggregation. It’s a very structured way to handle the complexity of having so many different kinds of devices involved.

Meng: From where I sit, what I find interesting is how they manage the resource allocation dynamically. It doesn't just throw everything at every client; it adapts based on how much processing power and bandwidth each device actually has available.

Lalam: I can confirm that the core idea is using this dynamic splitting strategy to balance the cost of computation on the device against keeping global convergence stable, which is a big challenge when you’re dealing with non-IID data across a network.

Tom: And they tackle that heterogeneity by using these behavior-aware fingerprints. Instead of just grouping clients by what their labels are, they look at how their models actually behave when shown some public inputs to figure out which ones are similar.

Jane: So it’s like the AI is figuring out, "Hey, this client is acting exactly like that other client we saw on our probe data," which lets them trust the updates more.

Lu: That prediction-consistency trust scoring is key there; it basically tells you if a client’s update looks weird or polluted because their local data distribution is too different.

Meng: And they handle the communication cost with something called computational sketches and SS-OP perturbation to keep those model exchanges small, which helps with both speed and keeping the training private.

Lalam: The convergence analysis theorem they put up shows that under their assumptions, the global adapter parameters actually follow a predictable error bound as training goes on.

Tom: That’s solid because it means we aren't just guessing; there’s a mathematical guarantee on how fast this framework will reach a good solution.

Jane: It suggests that for deploying LLMs in real-world environments, you can get strong performance while keeping the communication overhead low and privacy concerns in check.

Lu: The main thing they caution about is that if you let the local blocks grow too large—like up to six blocks—the model starts to become too specialized for a single client’s data, and performance actually drops.

Meng: So it’s not just about how much work you offload; it's about finding that sweet spot where the model stays general enough to learn something useful across the whole network.

Tom: Yeah, so ELSA is a framework that systematically coordinates computation, manages heterogeneity through behavioral clustering, and controls communication with sketches and perturbation. It’s a comprehensive approach for edge LLM training.

Jane: And it really shifts the focus from just building bigger models to building smarter ways to train them when you can't afford massive centralized compute. (Music swells slightly)

Xiaohong Yang, Tong Xie, Minghui Liwang, Chikai Shang, Yang Lu, Zhenzhen Jiao, Liqun Fu, Seyyedali Hosseinalipour

School of Informatics, Xiamen University Department of Control Science and Engineering National Key Laboratory of Autonomous Intelligent Unmanned Systems Frontiers Science Center for Intelligent Autonomous Systems Ministry of Education Teleinfo iF-Labs University at Buffalo SUNY

cs.LG

Submitted: 2026-01-20

Updated: 2026-10-17

Comments: 28 pages, 8 figures

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

The gist: The gist The proposed ELSA framework systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks.

Key concepts

Split Learning (SL): This strategy divides the training of a large model into segments. The output layer stays on the client device for local training, while intermediate layers are sent to a central server. This allows clients to train locally without exposing raw gradients or logits to the server, balancing privacy and distributed computation.
Hierarchical Federated Learning (HFL): HFL structures the training process across multiple tiers: clients on the edge, edge servers, and a cloud aggregator. ELSA uses this hierarchy to coordinate computation efficiently. Clients handle initial local training tasks, while edge servers manage intermediate model segments, and the cloud aggregates final adapter updates.
Semantic Subspace Orthogonal Perturbation (SS-OP): This is a method used to compress and obfuscate model updates before transmission. It involves computing principal components of the hidden state and applying a random orthogonal matrix perturbation. This technique reduces the communication size significantly while ensuring that the transmitted data remains private and stable for training.
Behavior-aware Clustering: Instead of grouping clients by simple data labels, this mechanism groups them based on how their models behave when exposed to public inputs. It uses symmetric Kullback-Leibler divergence to measure semantic differences, helping to identify distinct local models despite data heterogeneity.

Terminology

Data Heterogeneity — 2× in this paper · explained in 3 episodes
Efficient LLM-centric Split Aggregation — 2× in this paper
Federated Learning — 2× in this paper · explained in 18 episodes
Fine-Tuning — 2× in this paper · explained in 5 episodes
Resource Constraints — 2× in this paper · explained in 5 episodes
Adaptability — 1× in this paper
Clustering — 1× in this paper · explained in 2 episodes
Communication Overhead — 1× in this paper · explained in 2 episodes
Consistency — 1× in this paper · explained in 3 episodes
Kullback-Leibler (KL) Divergence — 1× in this paper · explained in 6 episodes
Large Language Models (LLMs) — 1× in this paper · explained in 9 episodes
Robustness — 1× in this paper · explained in 6 episodes

Summary

The gist The proposed ELSA framework systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks.

How it works

ELSA is a novel framework that systematically integrates split learning (SL) and hierarchical federated learning (HFL) for distributed LLM fine-tuning over resource-constrained edge networks. It introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints using public probe inputs and symmetric Kullback-Leibler (KL) divergence. This mechanism is augmented by prediction-consistency trust scoring and latency-aware edge assignment to jointly mitigate data heterogeneity, device unreliability, and communication constraints. Second, it employs a resource-aware dynamic model splitting strategy to adaptively partition the LLM into three segments across clients and edge servers. The cloud is used only for adapter aggregation, enabling an effective balance between ondevice computation cost and global convergence stability. Third, it incorporates a lightweight communication scheme based on computational sketches combined with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead while mitigating privacy leakage during model exchanges across the network.

Behavior-aware Hierarchical Clustering and Robust Aggregation

The framework addresses data heterogeneity and unreliability by moving beyond label distribution-based clustering. It proposes a task-agnostic behavioral profiling framework that quantifies semantic behavioral divergence, identifying distinct local models through their differing predictive logic when exposed to the same inputs. This framework is built upon three pillars:

[Probe-based behavioral fingerprint derived from public inputs enables KLD-driven clustering without assuming task type or label space of clients]

[A trustworthiness score grounded in prediction consistency to down-weight clients with polluted or anomalous data]

[A client-to-server assignment policy that respects physical network constraints between clients and edge servers]

The process involves four steps:

Step 1: Construction of a Public Probe Set

Step 2: Task-Agnostic Behavioral Fingerprint Extraction

Step 3: Symmetric KLD for Behavioral Discrepancy

Step 4: Trust and Communication-Aware Client Clustering

Model Splitting and Collaborative Training

ELSA employs a tripartite model segmentation in its HFL framework. The design retains the output layer on the client side to eliminate label leakage during training and keeps both input embedding and output layers locally to prevent direct server access to raw gradients or logits. The intermediate transformer blocks are offloaded to the server for Part 2 of the training process.

The model splitting strategy is resource-aware and dynamic. It determines the number of layers for each segment adaptively for each client based on their available computational capacity Hn (e.g., FLOPS) and uplink bandwidth Bn. The offloading preference score Gn is defined as Gn = λ1 · (1 − Hn / Hmax) + λ2 · Bn / Bmax. This allows clients with limited resources or high-quality connections to offload more work to the edge, while powerful clients perform more local computation. The number of offloaded layers qn is then dynamically calculated as qn = M − ofix − pn.

Layered Compression for Secure and Efficient Communications

To reduce communication overhead and mitigate privacy leakage, ELSA introduces a layered compression-and-obfuscation mechanism based on computational sketches. Specifically, it proposes a semantic subspace orthogonal perturbation (SS-OP) mechanism. This involves computing the top-r principal components via truncated singular value decomposition (SVD) and generating a client-specific random orthogonal matrix Vn. The full perturbation matrix is then constructed as Qn = UnVnU⊤n + (IDhidden − UnU⊤n). This design ensures that privacy is strengthened and training remains stable.

The uploaded hidden state is then computed as He n = QnHn, which is compressed into a Y × Z sketch matrix U using Y independent hash functions. The compression ratio throughout the entire transmission process is defined as ρ = D/(Y Z).

Convergence Analysis

Theorem 1 establishes the convergence of ELSA. Under Assumptions 1–3 (in the Appendix A), the sequence of global adapter parameters θg generated by ELSA satisfies a bound on the optimization error. The bound is given by 1/G G X−1 g=0 E[∥∇Fe(θg)∥ 2] ≤ 4L(Fe(θ0) − Fe∗)√G + σ 2√local G + σ 2 2. This matches the bound in Theorem 1.

Experimental Evaluation

Extensive evaluations across eight diverse NLP benchmarks demonstrate that ELSA consistently outperforms state-of-the-art baselines in both learning efficiency and final performance. For instance, ELSA attains state-of-the-art results (e.g., 80.93% on RTE, 81.78% on MultiRC) on all benchmarks except SQuAD. Furthermore, ELSA consistently achieves the lowest communication latency in all scenarios. The results suggest that ρ ∈ [2.1, 4.2] offers a practical operating region that balances communication efficiency and learning performance.

Impact of Model Segmentation

The evaluation of model segmentation reveals that a significant performance degradation is observed as the number of local blocks increases to 6. This phenomenon is attributed to the fact that upon having a large number of training layers, it still has a certain impact, that is, the feature extractor becomes “too personalized” to the local (usually non-IID) data distribution and cannot capture a universal semantic representation. Building on this insight, we configure our resource-aware dynamic splitting strategy with an upper bound pmax = 6 to prevent performance degradation from over-personalization.

Conclusion

ELSA addresses the coupled challenges of computational fragmentation, data heterogeneity, and communication overhead. Extensive evaluations across eight diverse NLP benchmarks demonstrated that ELSA consistently outperforms state-of-the-art baselines in both learning efficiency and final performance. Future research directions include the extension of ELSA to ultra-large-scale models with substantially increased parameter counts, such as LLaMA- and GPT-scale architectures.

Improvements for AI systems

Behavior-aware client clustering improves model adaptation by quantifying semantic behavioral divergence beyond label statistics, allowing clients to be grouped based on differing predictive logic when exposed to the same inputs rather than just class frequencies. This allows for more accurate task-agnostic grouping across different NLP tasks where label spaces are not universal.
Resource-aware dynamic model splitting ensures optimal balance between computation cost and convergence stability by adaptively partitioning the LLM into segments based on client capabilities and network bandwidth, as determined by the offloading preference score Gn. This prevents over-personalization of local feature extractors, maintaining global model generalization while accommodating diverse hardware.
Privacy is significantly strengthened during model exchange through a layered compression and obfuscation mechanism called Semantic Subspace Orthogonal Perturbation (SS-OP). This technique ensures that Adversaries cannot recover the original semantics without knowing both Un (data-dependent) and Vn (secret-seeded), effectively mitigating information leakage during training.
The system can perform distributed fine-tuning of large language models across resource-constrained edge networks while maintaining high performance, as ELSA consistently outperforms state-of-the-art baselines in terms of adaptability, convergence behavior, and robustness. This enables the deployment of LLM fine-tuning on devices with limited compute and memory.
Communication overhead is drastically reduced through computational sketches combined with SS-OP, leading to a total communication time reduction of approximately 69.3% to 73.7% (e.g., from 10.51x10 cubed s to 3.22x10 cubed s on AG News) compared to uncompressed baselines under convergence conditions.

Abstract

Training large language models (LLMs) in resource-constrained edge networks poses fundamental challenges in computation, communication, data heterogeneity, and privacy. To address these challenges, we propose ELSA (Efficient LLM-Centric Split Aggregation), a distributed fine-tuning framework that integrates split learning (SL) with hierarchical federated learning (HFL) to coordinate computation and model aggregation across heterogeneous clients, edge servers, and the cloud. ELSA introduces three key innovations. First, it employs a task-agnostic, behavior-aware client clustering mechanism that constructs semantic fingerprints from public probe inputs using symmetric Kullback-Leibler (KL) divergence, complemented by prediction-consistency-based trust scoring and latency-aware edge assignment to account for data heterogeneity, client reliability, and communication constraints. Second, it adopts a resource-aware dynamic model partitioning strategy that distributes the LLM across clients and edge servers in three segments, while reserving the cloud for adapter aggregation. This design coordinates computation across heterogeneous nodes while balancing on-device computational demands and global convergence behavior. Third, it integrates computational sketches with semantic subspace orthogonal perturbation (SS-OP) to reduce communication overhead and mitigate privacy leakage during model exchanges. Extensive experiments across diverse NLP tasks demonstrate that ELSA outperforms state-of-the-art baselines in model performance, convergence behavior, and robustness. Further, on-device profiling on NVIDIA RTX 4090 GPUs and Jetson AGX Orin edge devices characterizes the memory footprint, energy consumption, and execution latency of key training stages, providing empirical evidence of ELSA's practical feasibility.