SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection

arXiv:2603.03022 · cs.LG · Submitted 2026-03-03 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection".

Jane: Here's the gist of the paper. The gist In recent years, multi-view multi-label learning (MVML) has attracted extensive attention due to its close alignment to real-world scenarios and information-theoretic methods have gained prominence for learning nonlinear correlations.

Tom: How it works The core idea of SEHFS is to convert the feature graph into a structural-entropy–minimizing encoding tree, quantifying the information cost of high-order dependencies and thus learning high-order feature correlations beyond pairwise correlations.

Jane: Specifically, features exhibiting strong high-order redundancy are grouped into a single cluster within the encoding tree, while intercluster feature correlations are minimized, thereby eliminating redundancy both within and across clusters.

Tom: Furthermore, a new framework based on the fusion of information theory and matrix methods is adopted, which learns a shared semantic matrix and view-specific contribution matrices to reconstruct a global view matrix.

Paper summary: Tom: So we're looking at this paper today, "SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection". It's tackling a big problem in multi-view multi-label learning.

Jane: That sounds complicated. So what’s the main thing they’re trying to solve?

Tom: They noticed that features in real life often have these high-order structural correlations, which are more complex than just simple pairwise relationships. Existing information theory methods struggle to find those high-order connections, and they also tend to get stuck in local optima because of how they optimize things.

Lu: Exactly. The core idea here is to convert the feature graph into a structural entropy-minimizing encoding tree, which lets them quantify the information cost of these high-order dependencies beyond just pairwise correlations.

Meng: So, instead of just looking at two features together, they are trying to find patterns involving many features at once by minimizing this structural entropy. What does that actually mean for feature selection?

Jane: It means they're grouping features that have strong high-order redundancy into a single cluster in this tree, while making sure the correlations *between* those clusters are minimized. That way, they cut down on redundancy both inside and across those groups.

Lalam: I can confirm that the paper proposes using mutual information to explore relationships between data and labels, but that's usually limited to pairwise or low-order relationships fifteen, sixteen <ref:2603.03022#pg2,mutual information to explore relationships>.

Tom: Right. And SEHFS uses structural entropy via matrix operations to introduce a structural entropy regularization term. This converts the feature graph into that encoding tree structure, which is how they learn those higher-order feature correlations they mentioned earlier.

Jane: And it doesn't just stop there; they also use a new framework based on fusing information theory and matrix methods. They learn a shared semantic matrix and view-specific contribution matrices to rebuild a global view matrix.

Lu: That global view reconstruction part is important because it aims to get a unified feature matrix with cross-view consistency, which helps with overall feature selection across all views.

Meng: I’m thinking about the practical side here. How does this fusion of information theory and matrices help balance the global and local optimization objectives they mentioned?

Tom: Well, they formulate a final objective function that includes terms for both reconstruction accuracy and this structural entropy term, plus some regularization terms involving the semantic matrix and view-specific contribution matrices. That's all in one equation, subject to some non-negativity constraints.

Jane: It sounds like they are balancing the need for a consistent global view with the need to capture local details within each view simultaneously. So, what’s this whole process actually doing for a person who just uses AI models?

Lalam: This method is designed to take that reconstructed global matrix and then perform sample-level optimization by picking features from it. It’s a multi-step process.

Paper summary: Tom: The paper claims this approach outperforms eight state-of-the-art methods across eight different datasets from various domains. They say SEHFS achieved the best evaluation in eighty-seven point five percent of cases across four evaluation metrics, and for the Hamming Loss metric specifically, it got a perfect one hundred percent rate.

Meng: It’s good to see those results on multiple domains, but I wonder how robust that is when you move this onto real traffic at scale. Does the O T(n 2d+nd two) complexity make it too slow for production deployment <ref:2603.03022#pg2>?

Lu: The time complexity is stated as O T(n 2d+nd two), which scales linearly with T and polynomially with n and d <ref:2603.03022#pg2>. That suggests it's manageable in terms of scaling, though the polynomial part still depends on the feature dimension d.

Jane: So, to put this simply for a listener who isn't deep in math: they took complex features from different views, figured out how they relate using structural entropy to find deeper connections than just simple pairings, and built a unified view matrix that balances consistency across all views with the unique information each view contributes.

Tom: That’s the gist of "SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection". It’s about moving beyond what we can easily see in simple feature pairs to understand more complex data structures across multiple views.

Jane: The authors are showing that by using this structural entropy measure and the matrix fusion framework, they can learn those high-order dependencies without getting trapped in the local optima that other heuristic methods often fall into.

Lu: What it changes is how we think about feature selection in complex multi-view scenarios, moving from low-order correlation assumptions to a method that explicitly models structural entropy to capture higher-order redundancies.

Meng: For deployment, the key thing is that they’ve formalized this balance between global and local optimization into one framework, which makes the process more systematic rather than relying on trial and error with different algorithms.

Tom: So, in conclusion, SEHFS uses structural entropy to find high-order correlations and a matrix fusion framework to manage the trade-off between global consistency and view-specific details.

Jane: It’s about getting a better picture of how features interact in multi-view learning by explicitly modeling the structure of those interactions through this entropy measure.

Tom: So, to wrap up, "SEHFS: Structural Entropy-Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection" uses structural entropy as a new way to learn high-order nonlinear feature correlations and reduce redundancy in multi-view learning.

Jane: It fuses information theory with matrix operations to balance global and local generalization by reconstructing a global view matrix using a shared semantic matrix and view-specific contribution matrices.

Lu: This approach establishes the theoretical ability of structural entropy to learn high-order correlations, which was previously hard to achieve.

Meng: The practical implication is that we have a more structured way to select features in these complex systems without relying purely on heuristic search strategies that often fail at scale.

Tom: It’s about getting a better picture of how features interact across views by balancing consistency and complementarity through this new structural entropy guided approach.

Conclusion: Tom: So we’ve gone through how SEHFS uses structural entropy to find those high-order feature connections and then fused matrix operations to balance global and local optimization across different data views, right?

Jane: That’s a lot of jargon for what boils down to finding smarter ways to pick features when you have multiple different perspectives on the same data.

Lu: The authors are focusing on using structural entropy as a new measure for learning those complex, non-linear feature correlations, which is something existing information theory methods struggle with.

Meng: It seems like they’re trying to solve that problem of getting stuck in local optima because their optimization process isn't capturing the full structure of the data dependencies.

Lalam: The paper shows that by reconstructing a global view matrix using a shared semantic matrix and view-specific contribution matrices, they can achieve better consistency across those different views.

Tom: So it’s about getting that balance between what’s consistent everywhere and what’s unique to each view during the selection process.

Jane: And the results are pretty compelling because they tested this on eight different datasets from various domains and found it outperformed eight state-of-the-art methods in most cases.

Lu: The methodology is built on a new way to define encoding trees based on structural entropy, which quantifies the information cost of those high-order dependencies.

Meng: From a deployment standpoint, the fact that they formalized this fusion between information theory and matrix methods into one objective function is important because it makes the process more structured.

Tom: It shows that you can use these advanced mathematical tools to systematically reduce feature redundancy in multi-view learning without just guessing at what features are useful.

Jane: What this means for a listener is that if you’re building an AI system with multiple data sources, this method offers a way to ensure the features you select actually understand how they interact on a deeper level.

Lu: It establishes that structural entropy can be used effectively to learn those higher-order feature correlations we couldn't before.

Tom: So the authors are proving that modeling the structure of dependencies through entropy is a viable path for improving feature selection in this complex area.

Jane: And now they’re showing how it balances consistency across views with the unique information each view brings to the table.

Cheng Peng, Yonghao Li, Wanfu Gao, Jie Wen, Weiping Ding

Jilin University · Southwestern University of Finance and Economics · Harbin Institute of Technology, Shenzhen · Nantong University · City University of Macau

cs.LG

Submitted: 2026-03-03

Updated: 2026-10-20

Comments: Accepted to IEEE Transactions on Image Processing

Journal ref: IEEE Transactions on Image Processing, vol. 35, pp. 10972-10986, 2026

DOI: 10.1109/TIP.2026.3740274

Project page: https://sites.google.com/view/jerrywen-hit/home

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 82/100

The gist: The gist In recent years, multi-view multi-label learning (MVML) has attracted extensive attention due to its close alignment to real-world scenarios and information-theoretic methods have gained

Key concepts

Structural Entropy: This is a new measure used in SEHFS that quantifies the information cost of dependencies within the feature graph. It allows the method to learn high-order correlations beyond simple pairwise relationships by converting the feature structure into an encoding tree that minimizes this structural entropy, effectively identifying and eliminating redundant features.
Global View Matrix Reconstruction: In multi-view learning, this process aims to create a unified feature matrix that is consistent across different views. It is achieved by learning two components: a shared semantic matrix to capture consistency across views and view-specific contribution matrices to capture complementary information from each view.
Information-Matrix Fusion Framework: This framework combines information theory with matrix operations to optimize the overall system. It learns a shared semantic matrix and view-specific contribution matrices, which are then used to reconstruct the global view matrix. This fusion helps balance the need for global consistency with local optimization objectives.

Terminology

Feature Selection — 3× in this paper · explained in 4 episodes
Redundancy — 2× in this paper · explained in 2 episodes
Ablation — 1× in this paper · explained in 1 episode
Ablation Studies — 1× in this paper · explained in 1 episode
Core — 1× in this paper · explained in 2 episodes
Local Optimization — 1× in this paper · explained in 2 episodes

Summary

The gist In recent years, multi-view multi-label learning (MVML) has attracted extensive attention due to its close alignment to real-world scenarios and information-theoretic methods have gained prominence for learning nonlinear correlations.

How it works

The core idea of SEHFS is to convert the feature graph into a structural-entropy–minimizing encoding tree, quantifying the information cost of high-order dependencies and thus learning high-order feature correlations beyond pairwise correlations. Specifically, features exhibiting strong high-order redundancy are grouped into a single cluster within the encoding tree, while intercluster feature correlations are minimized, thereby eliminating redundancy both within and across clusters. Furthermore, a new framework based on the fusion of information theory and matrix methods is adopted, which learns a shared semantic matrix and view-specific contribution matrices to reconstruct a global view matrix.

Structural Entropy guided Feature Selection Regularization

Most existing information-theoretic methods use mutual information to compute correlations either between features or between features and labels, but mutual information is fundamentally limited to pairwise correlations and only can learn low-order feature correlations. SEHFS utilizes the information-theoretic concept of structural entropy via matrix operations, introducing a structural entropy regularization term which can convert the feature graph into a structural-entropy–minimizing encoding tree and thus learn high-order feature correlations and eliminate redundancy among features. The definition of encoding tree and structural entropy is presented as follows.

Global View Matrix Reconstruction

In multi-view multi-label learning, information-theoretic methods are challenged in optimizing data from different views, and global view matrix reconstruction is proposed to obtain a unified feature matrix with crossview consistency, facilitating global feature selection. The reconstruction of the global view matrix mainly consists of two components: (1) learning a shared semantic matrix to capture consistency; (2) learning view-specific contribution matrices to capture complementarity.

Information-Matrix Fusion Framework

To reconcile global and local objectives, SEHFS proposes a novel multi-view, multi-label framework that reconstructs a global view matrix to optimize the overall structure and, in conjunction with a structural entropy–guided feature selection module, performs sample-level optimization by selecting features from the reconstructed global matrix. The final objective function is formulated as min Xf,W,S,Hv,αv XfW − Y2F + αL T SE(W) + β Xf − S X V v=1 αvXvPv2F + S − X V v=1 αvSv2F + λ Tr(LY⊤S LY) + γ X V v=1 XfP⊤ v − XvHv2F, s.t. Xf,W, S, Hv, αv ≥ 0, W1q = 1d.

Optimization and Performance

The optimization details of the five variables are presented as follows. The total time complexity is O T(n 2d+nd 2), scaling linearly with T and polynomially with n and d. Experimental results on eight cross-domain datasets demonstrate that the proposed method outperforms eight state-of-the-art methods. SEHFS achieved the best evaluation in 87.5% of cases across four evaluation metrics, and for the Hamming Loss metric specifically, SEHFS achieved the best evaluation with a perfect 100% rate.

Conclusion

In this paper, SEHFS employs structural entropy as a new measure for learning high-order nonlinear feature correlations and reducing redundancy, while fusing information theory and matrix operations to balance global-local generalization. SEHFS reconstructs a global view matrix with a shared semantic matrix and view-specific contribution matrices to achieve a balance between consistency and complementarity among views. Extensive experimental results demonstrate the superiority of our method.

--- Page 1 ---

Conclusion

Improvements for AI systems

Structural Entropy Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection (SEHFS) can be applied to feature selection in multi-view scenarios by converting the feature graph into a structural entropy minimizing encoding tree, which quantifies the information cost of high-order dependencies. This allows the system to learn high-order feature correlations beyond pairwise correlations, effectively eliminating redundancy both within and across clusters.
Multi-view multi-label learning systems can achieve improved consistency and complementarity by reconstructing a global view matrix using a fusion framework that learns a shared semantic matrix and view specific contribution matrices. This reconstruction balances global and local optimization, addressing the challenge that existing information theoretic methods are prone to converging to local optima.
The proposed system can perform superior feature selection by learning high-order feature correlations and eliminating redundancy, as demonstrated by the ability of structural entropy to learn these dependencies even when explicit pairwise signals are absent in scenarios like Scenario 1 (S1).
The system can maintain high accuracy across different dependency structures, as shown by its performance in Scenario 2 (S2), where it effectively compresses redundant information into single clusters, leading to a smaller approximation gap between the structural entropy solution and the ground truth joint entropy.

Abstract

In recent years, multi-view multi-label learning (MVML) has attracted extensive attention due to its close alignment to real-world scenarios. Information-theoretic methods have gained prominence for learning nonlinear correlations. However, two key challenges persist: first, features in real-world data commonly exhibit high-order structural correlations, but existing information-theoretic methods struggle to learn such correlations; second, commonly relying on heuristic optimization, information-theoretic methods are prone to converging to local optima. To address these two challenges, we propose a novel method called Structural Entropy Guided High-Order Correlation Learning for Multi-View Multi-Label Feature Selection (SEHFS). The core idea of SEHFS is to convert the feature graph into a structural-entropy-minimizing encoding tree, quantifying the information cost of high-order dependencies and thus learning high-order feature correlations beyond pairwise correlations. Specifically, features exhibiting strong high-order redundancy are grouped into a single cluster within the encoding tree, while inter-cluster feaeture correlations are minimized, thereby eliminating redundancy both within and across clusters. Furthermore, a new framework based on the fusion of information theory and matrix methods is adopted, which learns a shared semantic matrix and view-specific contribution matrices to reconstruct a global view matrix, thereby enhancing the information-theoretic method and balancing the global and local optimization. The ability of structural entropy to learn high-order correlations is theoretically established, and and both experiments on eight datasets from various domains and ablation studies demonstrate that SEHFS achieves superior performance in feature selection.

Sources

Comprehensive Comparative Study of Multi-Label Classification Methods