A neural-network model to find correlations in astrochemical space

arXiv:2610.30842 · astro-ph.GA · Submitted 2026-10-22 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: We're looking at the paper: "A neural-network model to find correlations in astrochemical space".

Jocelyn: Here's the gist of the paper. A neural-network model to find correlations in astrochemical space establishes relationships between molecular structures and column densities in TMC-1 using astrochemically informed embeddings to identify promising candidates for future detection.

Vera: The detection of new interstellar molecules is a central objective of astrochemistry, and arguably the most transformative development in the field over the past decade has been the dramatic increase in both the number and diversity of detected species The article discusses how recent molecular detections have substantially reshaped our understanding of interstellar chemistry, in a transformative, rather than incremental, way The TMC-1 dark cloud is known to be one of the test beds for astrochemical models and the prototypical source for the detection of new molecules The methodology builds upon the general strategy introduced by Lee et al.

Jocelyn: (2021), which combines unsupervised numerical representations with supervised machine learning to establish relationships between molecular structure and observed abundances The first step in the machine-learning pipeline consists of encoding molecular structures into numerical representations suitable for machine learning using SMILES notation, a text-based language that encodes molecular structures as character strings These SMILES strings are then tokenized into sequences of chemical symbols, such as “C” for a carbon atom, “=” for a double bond, and “-” for a triple bond Molecular embeddings are constructed by applying natural language processing (NLP) techniques to these tokenized sequences through domain-adaptive pretraining (DAPT) of the ChemBERTa transformer model These embeddings provide a compact representation of molecular structure and composition and constitute the initial feature space for the supervised learning tasks performed in this work Model Architecture and Training The final supervised learning model arises from the combination of two types of features, the finetuned ChemBERTa molecular embeddings and physicochemical descriptors The architecture consists of two parallel branches that are later concatenated in a new layer, named “fusion” (subscript “fus”), to produce a combined representation of the molecule The model is constructed using Keras with Tensorflow as backend while training/validation split and standardization of the data is handled by scikit-learn Several regularization techniques are applied to the model, including neuron dropout, layer normalization, L2 regularization and Gaussian noise addition The optimization is carried out using Bayesian optimization routines implemented in Optuna (Akiba et al.

Vera: 2019) minimizing the following score function: Score = MSEval + 0.25σMSEval + 0.25MSEtrain, (1) Applications and Findings The model is used to explore large chemical databases, such as Zinc (Irwin et al.

Title and authors: Vera: So we're diving into the title and authors of "A neural-network model to find correlations in astrochemical space." It’s about using a neural network to find those correlations in astrochemical space. The authors are Vargas, Barreiro-Lage, Agúndez, Cabezas, Cernicharo, and Molpeceres from the Universidad Nacional de Colombia and IFF-CSIC in Madrid.

Jocelyn: It’s interesting because the title suggests this isn't just a simple correlation study; it implies a more complex relationship is being modeled between different molecular structures and their column densities. It’s about finding hidden structure in the data.

Subrahmanyan: From my perspective, that points toward using advanced machine learning techniques to uncover non-obvious chemical dependencies that might not be visible through traditional linear methods when you look at the data.

Vera: Exactly. They are taking molecular structures, encoded as chemical strings like SMILES, and trying to relate them directly to the observed column densities in TMC-one using this neural network pipeline <ref:2610.30842#pg1,molecular structures, encoded as chemical strings>.

Jocelyn: So it’s moving beyond just cataloging molecules; they’re building a predictive tool that learns the chemical relationships between what we see and how much of it there is.

Subrahmanyan: And that's significant because if you can map those relationships, you move from a descriptive inventory to a functional model of interstellar chemistry.

Vera: They are using this entire pipeline to establish these relationships, which is the core methodology they developed in line with approaches reported in the literature.

Jocelyn: So they’re applying state-of-the-art NLP models to encode the molecular structures, and then layering on supervised learning to find those correlations.

Subrahmanyan: And that's how you get from raw chemical strings to a quantifiable relationship between structure and abundance, which is exactly what we need for theoretical modeling.

The paper's summary: Vera: The paper summarizes the overall strategy of this work, which is to combine unsupervised numerical representations with supervised machine learning to establish those relationships between molecular structure and observed abundances in TMC-one <ref:2610.30842#pg1>.

Jocelyn: Basically, they take the molecular structures, turn them into numerical representations through those NLP techniques, and then use supervised learning to find the correlations with column densities.

Subrahmanyan: So the goal is to see if there’s a statistical link between how a molecule is built and how much of it exists in that dark cloud.

Vera: They generate molecular embeddings using domain-adaptive pretraining on ChemBERTa, which is an architecture designed to learn chemical information directly from SMILES representations.

Jocelyn: That means these embeddings are supposed to be a compact numerical summary of the molecule’s structure and composition.

Subrahmanyan: And that compact representation should capture the essential chemical information needed for the subsequent prediction steps, which is key for connecting structure to abundance.

Vera: They then combine those embeddings with physicochemical descriptors in a multibranch architecture that they call fusion to create a combined molecular representation.

Jocelyn: So it’s not just one piece of data; it’s structural information plus physical properties, all fused together for a more comprehensive input into the final prediction layer.

Subrahmanyan: That fusion step is what makes the architecture stronger because it allows the model to leverage both types of information simultaneously for a single output.

Vera: The final supervised learning model then tries to predict column densities based on this combined representation, using Keras with Tensorflow as the backend.

Jocelyn: And they optimize that prediction by minimizing a specific score function involving MSE and sigma MSE terms to find the best fit for their data.

Subrahmanyan: It’s a standard approach, but when you apply it to this complex molecular space, it tests how well this specific neural network architecture handles the complexity of chemical relationships.

The paper's improvements: Vera: They suggest that improving the molecular representation learning involves using transformer-based chemical language models adapted to astrochemistry through domain-specific pretraining on a dedicated inventory like Chaos, which they say this leads to improved predictive performance on both in distribution and out of distribution molecular inventories.

Jocelyn: That means they’re fine-tuning the language model specifically for astrochemical problems, rather than just using a general NLP model. It’s making the embeddings much more relevant to what we're studying.

Subrahmanyan: If you tailor the training data like that, you should get better results because the embeddings will capture chemical nuances specific to interstellar chemistry that a general model would miss.

Vera: They also say their multibranch neural network architecture can perform joint learning from molecular embeddings and physicochemical descriptors, allowing complementary structural and molecular information to be integrated within a single supervised representation for more robust predictions.

Jocelyn: So they’re not relying on just one feature set; they’s using both the structural embeddings and the physical descriptors together in a fusion layer.

Subrahmanyan: That joint learning should lead to better generalization because the model isn't overly specialized to just one kind of input, which is important when you want it to work across different chemical regimes.

Vera: And they also show that using different historical epochs helps show that recent molecular detections have substantially reshaped our understanding of interstellar chemistry in a transformative, rather than incremental way.

Jocelyn: So the paper argues that we can use these models to assess how quickly new discoveries are changing the field by training them on different historical inventories.

Subrahmanyan: That’s a powerful way to gauge the impact of new detections on our chemical models by comparing predictions across those different epochs.

Conclusion: Vera: So, to wrap up, this paper shows how finetuning ChemBERTa on TMC-one data using heuristic transformations called Chaos creates a learned vector representation that helps us identify promising candidates for future detection by leveraging latent vectors as a primary output <ref:2610.30842#pg1>.

Jocelyn: It’s about getting those latent vectors that map molecules into a two-dimensional UMAP space, which allows you to visualize where the highly abundant molecules are located in that space.

Subrahmanyan: That provides a basis to understand chemically related molecules and helps rationalize why a given molecule is present in the ISM by looking at its position in this learned chemical map.

Vera: They also found that the model predicts several molecules known or expected to be abundant, like H2, CH4, C2H2, C2H4, and even benzene and naphtalene.

Jocelyn: The model also flags some closed-shell species with moderate to large electric dipole moments as promising observational targets across different chemical families.

Subrahmanyan: The overall implication is that we have a learned vector representation of the astrochemical space tailored to molecular column densities in this astronomical region.

Vera: They also noted that their model’s performance on models trained on the inventory available before the QUIJOTE and GOTHAM surveys shows a markedly poorer predictive performance for recently detected species than models trained on the two thousand twenty-one inventory <ref:2610.30842#pg1>.

Jocelyn: So, it confirms that recent molecular detections have substantially reshaped our understanding of interstellar chemistry in a transformative, rather than incremental way.

Subrahmanyan: To conclude this work on "A neural-network model to find correlations in astrochemical space," we’ve established a learned vector representation of the astrochemical space tailored to molecular column densities in this astronomical region.

D. G. Vargas, D. Barreiro-Lage, M. Agünndez, C. Cabezas, J. Cernicharo, G. Molpeceres

Universidad Nacional de Colombia Sede Bogota Facultad de Ciencias Observatorio AstronÞmic Nacional Institute of Physical Fundamental Instituto de Fisica Fundamental CSIC

astro-ph.GA

Submitted: 2026-10-22

Updated: 2026-10-23

Comments: Accepted for publication in Astronomy and Astrophysics

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: The gist: A neural-network model to find correlations in astrochemical space establishes relationships between molecular structures and column densities in TMC-1 using astrochemically informed

Key concepts

SMILES notation: This is a text-based language used to represent molecular structures as character strings. For example, it uses symbols like 'C' for carbon or '=' for a double bond. This format allows the computer to translate chemical drawings into sequences that can be processed by machine learning models.
ChemBERTa embeddings: These are numerical representations of molecular structures created by applying Natural Language Processing (NLP) techniques to SMILES strings. They capture the essential chemical composition and structure of a molecule in a compact mathematical form, serving as rich input features for the main prediction model.
Latent Space Projection (UMAP): The model uses UMAP to reduce the high-dimensional molecular data into a two-dimensional map. This allows researchers to visualize the astrochemical space and see how molecules are grouped based on their abundance, helping identify regions where highly abundant or chemically related molecules reside.

Summary

The gist: A neural-network model to find correlations in astrochemical space establishes relationships between molecular structures and column densities in TMC-1 using astrochemically informed embeddings to identify promising candidates for future detection.

Introduction and Motivation

The detection of new interstellar molecules is a central objective of astrochemistry, and arguably the most transformative development in the field over the past decade has been the dramatic increase in both the number and diversity of detected species The article discusses how recent molecular detections have substantially reshaped our understanding of interstellar chemistry, in a transformative, rather than incremental, way The TMC-1 dark cloud is known to be one of the test beds for astrochemical models and the prototypical source for the detection of new molecules

Methodology

The methodology builds upon the general strategy introduced by Lee et al. (2021), which combines unsupervised numerical representations with supervised machine learning to establish relationships between molecular structure and observed abundances The first step in the machine-learning pipeline consists of encoding molecular structures into numerical representations suitable for machine learning using SMILES notation, a text-based language that encodes molecular structures as character strings These SMILES strings are then tokenized into sequences of chemical symbols, such as “C” for a carbon atom, “=” for a double bond, and “-” for a triple bond Molecular embeddings are constructed by applying natural language processing (NLP) techniques to these tokenized sequences through domain-adaptive pretraining (DAPT) of the ChemBERTa transformer model These embeddings provide a compact representation of molecular structure and composition and constitute the initial feature space for the supervised learning tasks performed in this work

Model Architecture and Training

The final supervised learning model arises from the combination of two types of features, the finetuned ChemBERTa molecular embeddings and physicochemical descriptors The architecture consists of two parallel branches that are later concatenated in a new layer, named “fusion” (subscript “fus”), to produce a combined representation of the molecule The model is constructed using Keras with Tensorflow as backend while training/validation split and standardization of the data is handled by scikit-learn Several regularization techniques are applied to the model, including neuron dropout, layer normalization, L2 regularization and Gaussian noise addition The optimization is carried out using Bayesian optimization routines implemented in Optuna (Akiba et al. 2019) minimizing the following score function: Score = MSEval + 0.25σMSEval + 0.25MSEtrain, (1)

Applications and Findings

The model is used to explore large chemical databases, such as Zinc (Irwin et al. 2020), to identify promising candidates for future detections or for astrochemical modelling The primary application involves identifying molecules deemed as astrochemically related as a preliminary step towards the identification of promising candidates for future detections A key finding is that the model identifies typical interstellar molecules such as H2, CH4, C2H2, C2H4, CH3+, CO2 or unsubstituted PAHs The latent vectors extracted from the last hidden layer of the supervised model allow for dimensionality reduction using UMAP to generate two-dimensional representations of the molecular inventory This task-specific latent space exhibits a clear separation according to molecular abundance, which is potentially the most impactful aspect of our work

Limitations and Conclusions

The main limitations of the model include the inclusion of information on non-detected molecules in the model, which would allow us to improve the regression of low abundance molecules A second limitation is the absence of an a posteriori estimation of prediction uncertainties, as it is the case with Bayesian approaches The results show that while performance improves with a larger training set, there are still significant outliers whose presence cannot be predicted a priori The model provides a learned vector representation of the astrochemical space tailored to molecular column densities in an astronomical region Future developments will focus on enhancing the ability of the model to capture the threedimensional structure of molecules and incorporating non-detections into the training dataset

How it works

The workflow generates astrochemically informed embeddings that can be employed in subsequent supervised learning tasks The latent vectors learned by the model represent a rich mathematical object that captures not only the underlying chemical relations, but also the potential interstellar detectability of the molecule The model achieves mean squared error values of 0.17/0.58 and mean absolute error values of 0.29/0.59, both expressed in logarithmic units of the column density

Model Training Details

The training procedure involves training several models using different random train–validation splits and reporting the average of their predictions The results obtained using three independent instances of the same model, with identical optimized hyperparameters (Table 1), but different random initializations and training–validation splits, are shown in Figure 3 The averaged metrics yield an MSE of 0.17±0.03, an MAE of 0.28±0.01, and an R2 (coefficient of determination) value of 0.89±0.02 for the combined training and evaluation set

Key Observations on Model Behavior

The model consistently overpredicts its abundance by two orders of magnitude in validation and by about one and a half orders of magnitude in training plus validation for l-C3H+ The exceptional reactivity of l-C3H+ is therefore a rarity that cannot be learned by the model, there is simply not enough information available to capture such an exception The overprediction of the small anion C4H–, and the underprediction of the large anion C10H– finds its origin in the well known increment in reactivity with increasing size The model's performance on molecules discovered after a certain inventory cutoff date shows that these species contain genuinely novel chemical information that was not already encoded in the previously known molecular inventory

Future Directions

The future developments of the model shown in this work will focus on enhancing the ability of the model to capture the threedimensional structure of molecules We also plan to investigate rigorous strategies for incorporating non-detections and observational upper limits into the training dataset Finally, we will explore the application of the model to astronomical sources with markedly different chemical compositions, including oxygenrich environments, in order to assess its generality and predictive power across diverse astrochemical regimes Column density guided latent representations such as those developed here could provide a complementary source of astrochemical information for automated spectral-analysis frameworks such as that of Fried et al. (2026)

Conclusion

The main points derived from our work are the following – We have finetuned a pretrained chemical language model, ChemBERTa, on molecules derived from the TMC-1 inventory, using heuristic transformations to generate new molecules through a protocol that we named Chaos We constructed a multibranch neural network architecture that combines the ChemBERTa fine tuned embedding with physicochemical molecular descriptors Interestingly, for most of the worst behaving points in the prediction of molecular abundances within our dataset we can rationalize the reason for the failure a posteriori, as they correspond to well molecules with well known peculiar chemical properties, such as l-C3H+, C4H– or H2C3O We have applied our methodology to the analysis of recent molecular detections in TMC-1 using three models corresponding to different stages in the characterization of its molecular inventory The model trained on the inventory available before the QUIJOTE and GOTHAM surveys shows a markedly poorer predictive performance for recently detected species than models trained on the 2021 inventory The resulting ensemble of three models show performances comparable to those discussed in Section 3 using an ensemble of three models

Selected Candidates

The predicted set also contains several molecules that are known or expected to be abundant in the interstellar medium, but remain difficult to observe because of strong depletion onto dust grains, such as H2O and HNO The model exhibits a noticeable bias toward anions and carbene/carbyne-like species, which we attribute to the important representation of large in size neutral and anionic polyynes in the TMC-1 inventory used during training Among those molecules, we identified typical interstellar molecules such as H2, CH4, C2H2, C2H4, CH3+, CO2 or unsubstituted PAHs The model further predicts the unsubstituted polycyclic aromatic hydrocarbons benzene and naphtalene We also identify some closed shell species with moderate to large electric dipole moments that constitute some promising observational targets across different chemical families

Data Availability

The code used to generate the vector representations of the molecules, list of smiles, and training routines are provided in the following repository https://github.

Improvements for AI systems

Improved molecular representation learning involves using transformer-based chemical language models adapted to astrochemistry through domain-specific pretraining (DAPT) of ChemBERTa on a dedicated inventory like Chaos, as this leads to improved predictive performance on both in distribution and out of distribution molecular inventories.
The resulting multibranch neural network architecture can perform joint learning from molecular embeddings and physicochemical descriptors, allowing complementary structural and molecular information to be integrated within a single supervised representation for more robust predictions.
The system can be used to identify promising candidates for future detection by leveraging latent vectors as a primary output, enabling further processing, e.g. clustering or classification, rather than just scalar abundance predictions, which helps in identifying several promising candidates for future detection in this cloud.
The model can perform a task-specific representation of the astrochemical space by using latent vectors to project new molecules into a two-dimensional UMAP space and identifying regions where highly abundant molecules are located, thus providing a basis to understand chemically related molecules and to rationalize the reasons for the presence of a given molecule in the ISM.
The system can assess the impact of recent molecular detections by training models on different historical epochs, showing that recent molecular detections have substantially reshaped our understanding of interstellar chemistry, in a transformative, rather than incremental, way.