ARISE: An adaptive residual-informed stability ensemble for feature selection in small-sample biomedical omics

The gist ARISE, an adaptive residual-informed stability ensemble, proposes a novel feature-selection framework for small-sample biomedical omics that integrates multiple complementary relevance signals with class-balanced stability assessment and residual-informed redundancy control to identify predictive, stable, and nonredundant feature subsets. This method is designed to address the challenges of large p, small n regimes in molecular classification by jointly managing relevance, stability, redundancy, and multiclass discrimination

Objective and Framework

The objective of ARISE is to develop a feature-selection method that can handle molecular classification involving a small number of observations by identifying predictive, stable, and nonredundant feature subsets applicable to both binary and multiclass outcomes. The proposed ARISE method integrates several complementary signals: multiple complementary relevance signals through fifteen predefined selector profiles whose contributions are adaptively weighted using nested inner crossvalidation. The framework combines seven percentilenormalized relevance components and evaluates performance across five molecular datasets, eight feature-set sizes, three fixed classifiers (k-nearest neighbours, support vector machine, and random forest), and six standalone filter-based comparators.

Key Components of ARISE

The ARISE framework is constructed from several interconnected modules that address different aspects of feature selection:

1. Relevance Signals: The method combines seven percentilenormalized relevance components through fifteen predefined selector profiles. These components are derived from various statistical measures, including POS, Fisher score, signal-to-noise ratio (SNR), MI, Bhattacharyya distance, and Welch score.

2. Stability Assessment: The framework estimates class-balanced subsample robustness by drawing B = 8 independent draws, m observations are sampled without replacement from every class to recompute components. This results in a subsampleaveraged robustness score Sgc which is then combined with relevance to form the Combined score Agc = (1 − αc)Rgc + αcSgc.

3. Redundancy Control: The method penalizes within-class correlation by calculating redundancy after within-class centering using a term ρg(S) which is based on the within-class scale. This redundancy penalty is incorporated into the greedy path selection equation (3).

4. Pair Coverage: For multiclass discrimination, ARISE rewards features that resolve different class contrasts through a term Fc(S) which is defined as the profile-c weighted pairwise utility for pair m. This coverage gain is added to the objective function in equation (3).

Methodology and Evaluation

The ARISE training and nested feature-path construction process involves a complex, multi-stage procedure:

1. Profile Racing: The framework utilizes three-fold inner cross-validation performs two-stage racing to select the best profiles from the library of 15, with Stage 2 evaluating those three profiles at all eight budgets and three classifiers.

2. Consensus Selection: After profiling, the best-plus-one-SE rule is recomputed among the three survivors to derive soft weights ωc by equation (5).

3. Greedy Path Generation: The final output is one ordered path; all feature budgets are exact prefixes generated by the greedy routine using equation (3).

The evaluation involved 210,000 held-out assessments across five molecular datasets, eight feature budgets, and three fixed classifiers. The results showed that ARISE achieved the highest overall performance for all three evaluation metrics across all 15 dataset–metric combinations.

Conclusion

ARISE yields a transparent and adaptive framework for molecular feature selection that jointly addresses relevance, stability, redundancy, and multiclass discrimination. Its consistent performance across the five molecular datasets supports its potential for small-sample molecular classification. The method's value lies in how it reconciles signal geometries rather than in a new isolated score.

The paper is intended as a methodological evaluation rather than evidence of clinical deployment. The study demonstrates that the construction is bounded, the coverage term is monotone submodular, redundancy is invariant to class-specific shifts, paths are deterministically nested, and consensus parameters remain within the eligible-profile hull. The results support ARISE as a robust feature-selection framework. The study highlights that biomarker-level interpretation should be complemented by external validation and assay-level replication. The heterogeneity of the five datasets supports method development while preventing the pooled mean from being read as the expected effect for a single clinical population. The optimum ranged from 15 to 35 by dataset, arguing for budget choice inside the training loop when a study has no prespecified assay size. The study defines a rigorous benchmarking stage and provides a clear basis for future evaluation on prospectively specified datasets.

Data availability is reported through the data resources and originating studies identified in table 1 and section 3.1. The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Funding was provided by the United Arab Emirates University under grand 12B086. The CRediT authorship contribution statement details the roles of each author in the work, such as Zardad Khan's conceptualization and methodology. The references provided support the various techniques and related works discussed throughout the paper. The study concludes that ARISE provides a coherent multiclass feature-selection framework combining complementary relevance evidence, balanced-subsample robustness, within-class residual redundancy control, class-pair coverage, nested profile racing, and convex consensus. The paper emphasizes that the observed advantage is not confined to a single performance criterion. The study concludes that accurate prediction and reproducible biomarker identification are separate objectives. Future work should use chance-corrected stability, pathway coherence, assay replication, and untouched cohorts rather than interpreting selection frequency alone. The study concludes that ARISE is deliberately more transparent than graph or foundation models. The study concludes that a clinically credible extension should freeze the selector before testing, compare it with stable and network-based methods on untouched cohorts, and quantify calibration, decision-curve utility, assay feasibility, and the reproducibility of individual selected features. The study concludes that the repeated five-fold design provided extensive resampling across 250 held-out folds per condition. The study concludes that the dataset, rather than the individual fold, remained the unit of across-study inference. The study concludes that variation in selected-set overlap across datasets highlights the ability of ARISE to identify alternative competitive feature subsets in high-dimensional settings. The study concludes that the stability results support ARISE as a robust feature-selection framework. The study concludes that accurate prediction and reproducible biomarker identification are separate objectives. The study concludes that future work should use chance-corrected stability, pathway coherence, assay replication, and untouched cohorts rather than interpreting selection frequency alone. The study concludes that ARISE is deliberately more transparent than graph or foundation models.

1. Bold header: Adaptive Feature Selection for Small-Sample Omics

ARISE provides a transparent and adaptive framework for molecular feature selection that jointly addresses relevance, stability, redundancy, and multiclass discrimination, enabling molecular classification involving a small number of observations by integrating seven percentilenormalized relevance components with class-balanced stability assessment and residual-informed redundancy control.

2. Bold header: Robustness Against Sampling Perturbations

The framework implements subsampling robustness by computing the subsampleaveraged robustness score (Sgc) and combining it with relevance, defined as Agc = (1 − αc)Rgc + αcSgc, which is designed to resist sampling perturbations in the small-sample regime.

3. Bold header: Optimized Feature Budget Selection

Instead of a single optimal feature count, ARISE allows for flexible selection by showing that the best-performing feature budget varied among datasets, indicating that no single subset size was uniformly optimal, meaning the system can select exact prefixes for all requested budgets based on dataset-specific performance maxima.

4. Bold header: Multiclass Discrimination Enhancement

The system explicitly rewards features that resolve different class contrasts through a term called pair-coverage gain, ensuring that the selection process is guided by explicit multiclass pair coverage.

Objective: Small-sample molecular classification requires feature selectors that identify predictive, stable, and nonredundant subsets for binary and multiclass outcomes. We propose ARISE (Adaptive Residual-Informed Stability Ensemble), which integrates complementary relevance signals, class-balanced stability assessment, residual-informed redundancy control, and multiclass pairwise coverage. Methods: ARISE combines seven percentile-normalized relevance components through 15 predefined profiles, adaptively weighted by nested inner cross-validation. It was evaluated on five molecular datasets, eight feature-set sizes, three fixed classifiers (k-nearest neighbours, support vector machine, and random forest), and six filter comparators. Generalization was estimated by five-fold outer cross-validation repeated 50 times using balanced accuracy, macro-F1, and Cohen's kappa. Results: Across 210,000 held-out assessments, ARISE ranked first in all 15 dataset-metric combinations. Equal-dataset means were 0.793 for balanced accuracy, 0.776 for macro-F1, and 0.725 for kappa, exceeding the strongest aggregate comparator by 0.022, 0.023, and 0.028, respectively. Performance remained strong across compact feature sets, although the optimal budget differed by dataset. Conclusion: ARISE provides a transparent, adaptive framework that jointly addresses relevance, stability, redundancy, and multiclass discrimination. Its consistent results across datasets, classifiers, metrics, and feature-set sizes support further evaluation for small-sample molecular classification.