Correlation of divergency: c delta. Being different in a similar way or not

The correlation-of-divergency coefficient, cδ, is a custom statistical measure designed to quantify the similarity of internal divergence patterns between two groups of paired values. The gist: cδ compares whether the way values differ within one group mirrors the way values differ within another.

The concept and formulation of cδ

cδ is a custom equation developed to measure the association between patterns of divergence within two groups of values (x and y) It compares how each value in x differs from all other values in x, and does the same for y, then assesses whether these patterns of divergence are related To calculate the numerator (signal), for each xi, sum the squared differences between xi and all other xj (j ≠ i), and do the same for yi Take the square root of each sum (for magnitude), multiply the results for xi and yi, and sum over all i The denominator is the product of the average internal divergences of both groups This ensures that the coefficient is scale-invariant.

Interpretation and behavior of cδ

A high value of the cδ coefficient indicates that the patterns of divergence within x and y are similar, such that when a value in x is distant from others in x, the corresponding value in y is likewise distant from others in y (being different in the same way). Conversely, a low value suggests that the patterns of divergence are dissimilar or unrelated (being different in a different, perhaps unique, way). The cδ coefficient is non-negative by construction because the numerator is a sum of products of non-negative terms and the denominator is a product of nonnegative averages. A completely diametrically opposed divergence structure will not be recognised by cδ as ‘different,’ because cδ does not produce negative values and because the pattern of divergence in sets X and Y will be the same, although coming from different sources.

Relation to other statistical measures

Table 1 contrasts the cδ statistic against standard correlation, dispersion, and distribution comparison methods across key theoretical and practical dimensions. The cδ coefficient does not measure direct association between paired values; instead, it compares the internal patterns of divergence or variability within each group. Unlike Pearson and Spearman, which assess direct association between paired values, cδ compares the internal patterns of divergence or variability within each group. The measure is concerned with whether the way values are spread out or dispersed within x mirrors the dispersion within y, rather than whether high values in x correspond to high values in y.

Limitations and robustness

The standard cδ (Eq. 4 and Eq. 5) uses squared differences (xi − xj)2, which amplify the effect of outliers quadratically. The influence function of cδ with respect to a contaminating point is unbounded and grows quadratically with distance from the data bulk, because the squared-difference kernel has this property. Gini’s mean difference (Table 1), which uses xi − xj rather than (xi − xj)2, has a linear influence function. Truly robust measures such as the median absolute deviation (MAD) or Qn estimator achieve 50% breakdown points. A rank-based cδ (replacing raw values with ranks before computation) would inherit the bounded-influence properties of rankbased methods such as Pearson’s.

Inference and application considerations

Currently, no closed-form Null distribution exists for cδ. The most natural and defensible approach to hypothesis testing for cδ would be permutation testing, following established practice for energy distance and MMD. For cross-dataset comparability, analysts should always report the normalised cδ / cδ max ratio and accompany it with permutation-based p-values. The measure is appropriate when the research question specifically concerns whether the internal variability structure of one dataset mirrors that of another.

Conclusion

The correlation-of-divergency coefficient (cδ) is a bespoke statistical measure designed to quantify the similarity of internal divergence patterns between two groups of paired values. It addresses a gap in statistical methodology by focusing on comparing dispersion structures rather than value associations. The measure is original and addresses a gap in statistical methodology, as it focuses on comparing divergence patterns rather than direct associations. It is highly sensitive to outliers (quadratic influence function, breakdown point is near 0), undefined for zero-variance groups, but winsorisation, or taking the absolute of cδ may take away much the concerns. Permutation testing and bootstrap resampling offer the most viable paths to inference. The correlation-of-divergency equation can be used to quantify and compare the internal patterns of divergence, such as spread, variability, or dissimilarity, between two groups of values in any context where the analyst deems such comparison meaningful.

--- Page 1 ---

The gist: cδ compares whether the way values differ within one group mirrors the way values differ within another.

How it works

cδ is a custom equation developed to measure the association between patterns of divergence within two groups of values (x and y) It compares how each value in x differs from all other values in x, and does the same for y, then assesses whether these patterns of divergence are related. To calculate the numerator (signal), for each xi, sum the squared differences between xi and all other xj (j ≠ i), and do the same for yi Take the square root of each sum (for magnitude), multiply the results for xi and yi, and sum over all i. The denominator is the product of the average internal divergences of both groups This ensures that the coefficient is scale-invariant.

Relation to other statistical measures

Table 1 contrasts the cδ statistic against standard correlation, dispersion, and distribution comparison methods across key theoretical and practical dimensions. The cδ coefficient does not measure direct association between paired values; instead, it compares the internal patterns of divergence or variability within each group. Unlike Pearson and Spearman, which assess direct association between paired values, cδ compares the internal patterns of divergence or variability within each group.

Limitations and robustness

The standard cδ (Eq. 4 and Eq. 5) uses squared differences (xi − xj)2, which amplify the effect of outliers quadratically. The influence function of cδ with respect to a contaminating point is unbounded and grows quadratically with distance from the data bulk, because the squared-difference kernel has this property <ref:2510.

1. Improve clustering validation by comparing internal cohesion or dispersion using c-delta, as it enables the comparison of the internal cohesion or dispersion of clusters identified through clustering analyses. This allows for a structural similarity assessment rather than just pairwise distance comparisons.

2. Enhance feature selection in machine learning by using c-delta as a feature to quantify the similarity of internal variance structures between two sets of features or samples. This helps identify subsets where the variability profile is consistent across different data partitions.

3. Develop a more robust method for comparing measurement outcomes in quantum systems by using c-delta to compare how the variability of quantum observables changes under different environmental conditions or noise models. This provides insight into how noise affects the spread of results between two systems.

4. Implement outlier-resistant comparison tools by using the absolute difference variant of c-delta, which has a linear influence function and is substantially better outlier resistance while retaining high efficiency for near-normal data. This mitigates the quadratic amplification of extreme values inherent in the standard squared-difference formula.

5. Create a directional analysis layer by decomposing c-delta into magnitude and direction, using Pearson or Spearman correlation between the divergence vectors Dx,i and Dy,i to determine if patterns are aligned or inverted. This resolves the limitation where c-delta cannot distinguish between similar and perfectly inverse divergence structures.

Matched sets may have unlike mean scores and spreads yet share the same way in which some scores stand apart. Mean gaps and raw-score correlations do not answer this question. Correlation of divergency, c delta, is put forward to fill that gap. For every score in each of two equally sized, matched sets, it finds how far that score lies from every other score in its own set. It then draws these gaps into one divergence score through a power mean. The Pearson correlation between the two lists of divergence scores is c delta, which lies between-1 and 1. A high positive value means that the same people or things stand out in both sets, no matter which way their raw scores point. A negative value means that a score far from the rest in one set tends to be matched with a score near the rest in the other, whilst a value near zero shows little linear link. As c delta stems from within-set gaps, it is unchanged by shifts in overall level or units. It can show whether the same people feel sharply shut out in matched neighbourhood sets, whether the same bacterial strains grow oddly under two drugs, or whether the same proteins stand out in two drug-binding fingerprints. It should sit alongside means and raw-score correlations rather than take their place: c delta does not show whether an outlier is high or low, nor why it arose. The power p sets the weight given to wide gaps. With p=1 it uses Gini distance, and with p=2 root-mean-square distance. Higher p gives more weight to the furthest gaps but cuts Kish's effective sample size. The raw scores and the choice of p must therefore be read with care.