DynFS-MoE: Dynamic Functional-Structural Mixture-of-Experts for Post-Traumatic Epilepsy Diagnosis

The gist

DynFS-MoE proposes a dynamic multimodal Mixture-of-Experts framework that integrates functional and structural connectivity through time-aware functional-structural encoding and classconditioned expert routing to address the challenge of early identification in post-traumatic epilepsy (PTE) after traumatic brain injury (TBI)

Introduction to the Problem

Post-traumatic epilepsy (PTE) is a severe complication of traumatic brain injury (TBI) and early identification remains challenging due to the complex structural and functional alterations it induces in the brain. Structural MRI can precisely detect microstructural damage, brain atrophy, and hippocampal or thalamic volume loss. Moreover, resting-state functional magnetic resonance imaging (fMRI) further reveals functional connectivity abnormalities associated with the transition from TBI to PTE, such as thalamocortical hyperconnectivity. However, most existing studies employ only structural MRI [6, 7] or fMRI individually [10–12], focusing on structural damage (such as lesion volume and hippocampal or thalamic atrophy) or functional connectivity abnormalities (such as hyperconnectivity patterns) [8, 13]. Unimodal models generally yield lower predictive accuracy compared to multimodal integration and cannot simultaneously capture temporal dynamics and spatial network information, resulting in insufficient explanatory power for the heterogeneous progression of PTE [14–17].

Framework Components

DynFS-MoE consists of three components: (1) a timeaware functional-structural encoder that extracts temporal fMRI and structural connectivity representations into refined modality patches through segment-wise fusion, (2) a mixture-of-experts layer composed of modality-specific experts and crossmodal experts, which capture both modality-specific characteristics and cross-modal interactions, and (3) a modality-class MoE (MCoE) routing module that dynamically assigns expert weights using a gating mechanism conditioned on classaware representations.

Time-Aware Functional-Structural Encoder

The encoder jointly models the temporal dynamics of fMRI and the structural information from dMRI. The fMRI sequence is partitioned into K = T/L nonoverlapping temporal segments of fixed length L as patches, where each modality patch represents an embedding of modality-specific temporal (fMRI) or structural connectivity information. Specifically, the fMRI temporal patches F = 1(1), F(2), [F(K)] with each F(k) ∈ R x L. For the structural modality, we obtain a compact representation by extracting the upper-triangular entries of the connectivity matrix, denoted as vec△(S) ∈ R R (R+1)/2, where vec△(·) collects elements Si,j satisfying 1 ≤ i ≤ j ≤ R. The projected unified modality patch space is P = f(1),..., f(K), s, with cardinality P = K + 1.

Modality-Specific Experts (MSoEs)

To capture modality-specific characteristics, the framework employs modality-specific MoEs operating on modality patch embeddings pi ∈ P, pi ∈ R D. Given the expert set E, and a gating function computes routing probabilities πj (pi) = Pθ(E = ej P = pi) = softmaxj (Wgpi). The modality-specific representation is calculated with a weighted combination of expert outputs zspec i = PJ j=1 πj (pi) ϕj (pi), which allows different experts to capture heterogeneous modality-specific patterns in functional dynamics and structural connectivity, thereby improving representation expressiveness prior to classconditioned routing in the next Section 2.4.

Modality-Class MoE (MCoE) with Class-Conditioned Routing

The MCoE module enables different classes to learn the most suitable modality representations according to their assigned modality combinations and expert specialization. The input to this module is the set of refined modality patch embeddings Z = 1i=1, where each zi ∈ R D incorporates crossmodal fusion and modality-specific processing from prior layers (Sections 2.2 and 2.3). Class-conditioned routing logits are computed as L = f(Z) ⊙ Φ ∈ R B x P×J×C. The dispatch weights Dijy = exp(Lijy) / P i′=1 exp(Li′jy), which aggregate patches for each expert per class, representing P(Pi Ej, Y = y). The expert input is a weighted sum augmented by class embedding cy ∈ R D with the class-conditioned expert input ujy and expert output ojy can be expressed as ujy = (X P i=1 Dijy · zi) + cy, ojy = ϕj (ujy),. To combine expert outputs into class-specific features, we compute combine weights Cjy = exp(L¯ P jy) / J j′=1 exp(L¯ j′y) via softmax over experts, using logits from meanpooled Z, and we can yield the fused feature hy = PJ j=1 Cjy ·ojy, where class-specific features are concatenated with modality-specific representations and passed to a final MLP classifier for predicting y.

Interpretability Regularizer

The training objective includes a brain interpretability regularizer maximizing conditional mutual information I(P; E Y) = H(P Y)−H(P E, Y). This is calculated using logits L, joint and marginal probabilities are P(Pi, Ej Y) = exp(Lijy) / PP i′=1 exp(Li′j′y), and class-conditioned mutual information (CMI) can be written as Iθ(P; E Y) = X C y=1 P(Y = y) X P i=1 X J j=1 Pθ(Pi, Ej Y = y) · log Pθ(Pi Y = y) · Pθ(Ej Y = y). The overall objective is to minimize the following loss over all model parameters θ: min θ E (F,S,y ∼ D" Lclsfθ(F, S), y − λ Iθ(P; E Y).

Experimental Results and Interpretability

DynFS-MoE achieves the best AUC across all three binary classification tasks and competitive ACC/F1, with the strongest improvements observed for HC vs. PTE and TBI vs. PTE. For HC vs. PTE, DynFS-MoE further achieves statistically significant improvements in both AUC and F1 (p < 0.001), with F1=0.81 outperforming the best baseline MultiViT (F1=0.74). Ablation studies show that class-conditioned routing consistently improves AUC across all tasks, with pronounced gains in TBI vs. PTE (0.79 vs. 0.58) and HC vs. PTE (0.81 vs. 0.72), attributed to its ability to adaptively allocate expert capacity per diagnostic class. Interpretability analyses reveal that PTE consistently exhibits elevated segment importance at early-to-mid temporal segments (notably S2–S4), with cross-modal importance curves peaking sharply at S2 in HC vs. PTE and S4 in TBI vs. PTE, suggesting that early functional-structural interactions are particularly informative for PTE identification. Fig. 2 (B) displays pairwise ROI interaction heatmaps for TBI and PTE across Segments 1, 4, and 7. While both classes share a prominent cerebellar-vermis interaction core, PTE uniquely exhibits progressive recruitment of frontal-occipital (Frontal Med Orb L, Cuneus, Occipital Sup/Inf) and auditory cortical regions (Heschl R) in later segments, reflecting the temporal evolution of thalamocortical hyperconnectivity and limbic-occipital disruptions that distinguish PTE from TBI.

Conclusion

DynFS-MoE provides an interpretable tool for PTE classification and could support future studies on early detection and risk stratification. By combining time-aware functional-structural encoding with classconditioned expert routing, the model adaptively integrates fMRI and dMRI information and captures class-specific brain patterns. This dynamic routing strategy overcomes limitations of static fusion methods and improves discrimination performance across multiple classification tasks. Moreover, interpretability analyses reveal meaningful ROI interaction patterns, such as thalamic hyperconnectivity and limbic disruptions, providing clinically relevant insights. This framework effectively captures class-dependent brain interaction patterns and provides an interpretable approach for PTE diagnosis and risk stratification

References

[1] Michael C Dewan, Abbas Rattani, Saksham Gupta, Ronnie E Baticulon, Ya-Ching Hung, Maria Punchak, Amit Agrawal, Amos O Adeleye, Mark G Shrime, Andrés M Rubiano

[2] GBDTB Injury et al. Global regional and national burden of traumatic brain injury and spinal cord injury

[3] Oliver Acosta and Jaclyn Barcikowski.

1. No static fusion baselines are used, as "most existing studies employ only structural MRI [6, 7] or fMRI individually [10–12], focusing on structural damage (such as lesion volume and hippocampal or thalamic atrophy) or functional connectivity abnormalities (such as hyperconnectivity patterns) [8, 13]. The improved system can capture the heterogeneous progression of PTE" by simultaneously modeling temporal dynamics and spatial network information.

2. Implementation of a time-aware functional-structural encoder allows the model to jointly model the temporal dynamics of fMRI and the structural information from dMRI, enabling refined modality patches through segment-wise fusion. This allows for learning representations where the projected unified modality patch space is P = [f(1),..., f(K), s], with cardinality P = K + 1.

3. Incorporation of a lightweight cross-modal fusion step, defined by the equation ˜f(k) = f(k) + α(k)W V s, adaptively modulates structural connectivity contribution to each functional patch. This allows the model to learn cross-modal interactions specifically tailored for PTE diagnosis.

4. The Modality-Class MoE (MCoE) module dynamically adjusts expert weights using a gating mechanism conditioned on class-aware representations. This enables the system to dynamically route modality patches to specialized experts, capturing heterogeneous modality dependencies between classes like HC, TBI, and PTE.

5. The training objective includes a brain interpretability regularizer maximizing the class-conditioned mutual information I(P; E Y), which encourages class-specific modality routing. This allows the system to produce meaningful regions of interest (ROIs) interactions, such as thalamic hyperconnectivity and limbic disruptions, providing clinically relevant insights.

Post-traumatic epilepsy (PTE) is a severe complication of traumatic brain injury (TBI). Yet, early identification remains challenging due to the complex structural and functional alterations it induces in the brain. To address this, we propose a dynamic multimodal Mixture-of-Experts (MoE) framework that integrates functional and structural connectivity through time-aware functional-structural encoding and class-conditioned expert routing. Within this framework, modality-specific and cross-modal experts learn complementary representations, while a Modality-Class MoE (MCoE) module dynamically adjusts expert weights according to each classification objective. Experimental results across three binary classification tasks demonstrate that the framework consistently outperforms static fusion baselines, and high-interpretability analyses further reveal meaningful regions of interest (ROIs) interactions. This dynamic multimodal expert framework effectively captures class-dependent brain interaction patterns and provides an interpretable approach for PTE diagnosis and risk stratification.