Can phenotypic activity be predicted without experimental readouts?

The gist The molecular encoders CLOOME and CellCLIP show no clear advantage over plain physicochemical descriptors once leakage and toxicity confounds are controlled for, suggesting they should not yet be trusted as surrogates for phenotypic drug discovery.

Experimental Setup and Data

The study evaluates molecular encoders contrastively pretrained on paired molecule-morphology data, such as CLOOME and CellCLIP, to see if they can predict phenotypic activity without running a Cell Painting assay The evaluation protocol is designed to control for two confounds that can inflate apparent performance: leakage across an encoder’s own pretraining boundary, and the correlation between phenotypic activity and cytotoxicity Activity labels are generated using copairs permutation testing on the morphological profile, while toxicity labels are obtained from cell count alone restricted to compounds already called active The data utilized includes BBBC036v1, a curated U2OS screen with 10,680 compounds of which 91.3 % are active and 6.3 % are additionally toxic A larger multi-laboratory screen, JUMP-CP, is also used where 47.3 % of active JUMP-CP compounds are additionally called toxic

Molecular Representations and Controls

Six representations were tested for prediction targets:

1. MLP: a non-pretrained MLP control matching CLOOME’s input and layer count, trained directly and only on supervised labels

2. MorganFP: a radius-2, 2048-bit Morgan fingerprint used as a linear-probe input

3. PhysChem: the full RDKit descriptor set

4. LogP: a single Crippen logP value kept as its own representation to isolate the physicochemical property most classically linked to nonspecific bioactivity and cytotoxicity

5. CLOOME: a 4-layer multilayer perceptron mapping chiral Morgan fingerprints to 512-dimensional embeddings, trained via contrastive learning (CL) on BBBC036v1 moleculemorphology pairs

6. CellCLIP: a BERT-basecased text encoder with a linear projection to 512 dimensions, trained via CL on paired Cell Painting images and text

Evaluation Protocol and Findings

The representations were evaluated under three distinct splits: Chemical, Batch, and Official. Statistical comparisons used 5×-repeated, 5-fold grouped CV compared pairwise via repeated-measures ANOVA and Tukey’s HSD test for chemical and batch splits The official split is approximated by bootstrap-resampling held-out predictions Key findings indicate that once confounds are controlled for, the pretrained molecular encoders show no clear advantage over plain physicochemical descriptors Specifically, on BBBC036v1’s official split, PhysChem (0.568) and MLP (0.561) have heavily overlapping confidence intervals indistinguishable from CLOOME’s or CellCLIP’s own test AUROC (0.573 and 0.565; Figure 2)

Generalization Across Datasets

The results were validated across two Cell Painting resources with very different scale, chemical diversity, and laboratory origin to test generalization beyond a single screen On JUMP-CP’s chemical split (b), PhysChem is clearly best (0.704), followed by LogP, CellCLIP, MorganFP, and MLP (0.664 down to 0.653); CLOOME is last (0.626) with no overlap with any other representation The pattern replicates on the third population of JUMP-CP’s bioactive-library sub-population, where PhysChem is again clearly best on both its chemical (0.631) and plate (0.637) splits, and CLOOME is again at the bottom (0.549 and 0.545), tied with MorganFP for last

Conclusion on Surrogate Use

Toxicity is generally easier to predict than phenotypic activity across representations Active-not-toxic is the hardest target on BBBC036v1’s official split for every representation The paper concludes that activity-prediction results that do not separate the two should be read with caution, and confound and leakage-aware evaluation needs to become a basic reporting standard before such models can be trusted as surrogates for cellular response The authors believe these encoders are not yet ready to drive phenotypically-driven molecular generation.

Limitations and Future Work

Limitations include the fact that CLOOME and CellCLIP are both pretrained on BBBC036v1, making it hard to separate a general property of contrastive phenotype-pretraining from a property of this one, narrow pretraining set Furthermore, every representation is scored through a frozen linear probe, meaning only linearly accessible information is tested Future work should explore non-linear probes or fine-tuning. The effect of concentration remains unexplored since both activity and toxicity are dose-dependent The molecular-generation argument is a hypothesis to be tested in future work

Conclusion

The two tested encoders should not yet be trusted as phenotypic surrogates, but similar claims for these or future encoders should be checked under a comparable leakage-aware, confound-controlled evaluation The paper emphasizes that an encoder should be evaluated the way we evaluate phenotypic profiles themselves: on compounds and assays it could not have seen The final conclusion is that neither beats physicochemical descriptors that use no phenotypic data at all.

Author Contributions

TC contributed Conceptualization, Data curation, Methodology, Software, Formal analysis, Investigation, Validation, Visualization, Writing — original draft RM contributed Conceptualization, Methodology, Supervision, Funding acquisition Resources Writing — review & editing

Acknowledgments and Disclosure

TC and RM acknowledge funding provided by the Wallenberg AI, Autonomous Systems, and Software Program (WASP), supported by the Knut and Alice Wallenberg Foundation Computational resources were provided by the National Academic Infrastructure for Supercomputing in Sweden (NAISS), funded by the Swedish Research Council

Software and Data Availability

Code is available at https://github.com/cctelio/predicting-phenotypic-activity The full analysis, from raw data to every figure and table in this paper, reruns in a couple of hours on a consumer computer The references listed support the claims made throughout the paper The detailed methodology is described across several sections of the paper, including Appendix A, B, and C The statistical comparison details are provided in Section C The results are summarized in Figures 2, 3, 4, and 5 The entire analysis is reproducible at https://github.com/cctelio/predicting-phenotypic-activity The paper is available on arXiv as 2610.07997v1 [cs.

1. Focus on developing leakage-aware, confound-controlled evaluation as a standard practice before trusting phenotype-pretrained encoders as surrogates for phenotypic drug discovery, ensuring that performance gains are not artifacts of pretraining boundary leakage or correlation with cytotoxicity.

2. Implement a labeling scheme that statistically separates phenotypic activity from toxicity using the same validated permutation-test framework, yielding disentangled activity and toxicity labels (e.g., 'active, not toxic'). This allows for more nuanced predictions by isolating the biological response from non-specific cell death signals.

3. Design a robust leakage-aware evaluation protocol comparing chemical and batch-generalization splits against their own official pretraining-respecting split, as this prevents models from exploiting compounds seen during pretraining for better performance on unseen data.

4. Prioritize physicochemical descriptors like PhysChem and LogP when predicting activity, as the paper found that PhysChem matches or beats both pretrained encoders in certain settings, suggesting these baselines offer reliable structure-based predictions when phenotype-specific signals are weak.

5. Use the MLP control as a benchmark to assess whether pretraining provides genuine benefit; the finding that a not-pretrained control matching CLOOME’s input and layer count generalizes better than CLOOME itself suggests that end-to-end training on specific data may overfit to narrow chemical space.

6. For future generative models, avoid exploiting toxicity and not an interesting phenotype as a reward signal; instead, focus on generating compounds that exhibit genuine phenotypic changes rather than those that only score well due to similarity to the pretraining set or toxicity proxies.

Molecular encoders contrastively pretrained on paired molecule-morphology data, such as CLOOME and CellCLIP, have been proposed as cheap surrogates for phenotypic prediction, avoiding the need to run a Cell Painting assay. We evaluate this idea for these molecular encoders under a protocol designed to control for two confounds that can inflate apparent performance: leakage across an encoder's own pretraining boundary, and the correlation between phenotypic activity and cytotoxicity. Testing six representations, including a non-pretrained MLP control matching CLOOME's input and layer count, on two distinct Cell Painting screens, we find that once these confounds are controlled for, the pretrained molecular encoders show no clear advantage over plain physicochemical descriptors, and that toxicity is generally easier to predict than phenotypic activity across representations. Our results suggest leakage-aware, confound-controlled evaluation should be standard practice before phenotype-pretrained encoders are trusted as surrogates for phenotypic drug discovery.