Redshift Assessment Infrastructure Layers (RAIL): Rubin-era photometric redshift stress-testing and at-scale production

arXiv:2505.02928 · astro-ph.IM, astro-ph.CO, astro-ph.GA · Submitted 2025-05-05 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: Astrophysics Radio. Generated commentary on the latest astrophysics papers.

Vera: Hi, I'm Vera.

Jocelyn: And I'm Jocelyn, here with Subrahmanyan.

Vera: We're looking at the paper: "Redshift Assessment Infrastructure Layers (RAIL)".

Jocelyn: Here's the gist of the paper. The RAIL team presents an open-source Python library designed for at-scale probabilistic photo-z estimation, which serves as a flexible toolkit for stress-testing and deriving photo-z data products for extragalactic science.

Vera: RAIL's Core Functionality 1.

Jocelyn: The RAIL software library provides three subpackages offering modular tools for end-to-end stress-testing, including a "forward modeling suite to generate realistically complex photometry", a "unified API for estimating per-galaxy and ensemble redshift PDFs by an extensible set of algorithms", and "built-in metrics of both photo-z PDFs and point estimates" (Page 1).

Vera: 2.

Title and authors: Vera: Let's talk about the title and the authors for a minute. The paper is called "Redshift Assessment Infrastructure Layers (RAIL): Rubin-era photometric redshift stress-testing and at-scale production".

Jocelyn: The authors are a big team from various institutions, which tells you this is a collaborative effort pulling together different kinds of astronomical expertise, from survey photometry to pulsar timing.

Subrahmanyan: Having that mix of observational and theoretical backgrounds suggests they are aiming for a very comprehensive tool that bridges the gap between what the surveys see and what cosmology demands.

Vera: What this means in practice is they're building something open-source, like we discussed before, to help everyone test their photo-z methods against realistic LSST data scenarios.

Jocelyn: It’s about making sure that when you get those redshift numbers from the survey data, you know exactly how much noise and uncertainty is attached to them.

Subrahmanyan: So instead of just getting a single number for redshift, we get a whole distribution of possibilities for where that galaxy might be in the universe.

The paper's summary: Vera: Now, looking at the summary section of this paper, it talks about how photo-z estimates are subject to many sources of inaccuracy and imprecision, especially for the faint sources LSST is going to see.

Jocelyn: It emphasizes that because spectroscopic follow-up time is limited for the sheer number of galaxies we expect, we can't just rely on those confirmed redshifts anymore.

Subrahmanyan: That’s why they argue that photo-z probability density functions, or PDFs, are the favored tool because they comprehensively characterize that uncertainty better than a single point estimate does.

Vera: They review how to characterize these uncertainties, referencing work like Newman and Gruen from twenty twenty two to give us a solid foundation on what we should be looking for in these distributions.

Jocelyn: Basically, the core idea is that we need robust ways to generate and test those PDFs because they are the actual data product that tells us about our uncertainties.

Subrahmanyan: So it’s not just about getting a better point estimate; it’s about understanding the shape of the error itself across an entire sample of galaxies.

The paper's improvements: Vera: The paper then details the improvements they introduce to this infrastructure, which is where they introduce RAIL functionality. It outlines a structure with creation, estimation, and evaluation modules.

Jocelyn: They’ve put together these core building blocks so that researchers can run experiments in a traceable way using directed acyclic graphs or DAGs to connect all the stages.

Subrahmanyan: The main improvement is this modularity; it lets you swap out different components easily, whether you want to change how you create mock data or which estimation algorithm you use next.

Vera: They specifically mention the creation section where they have engines like FSPS to generate galaxy spectral energy distributions, incorporating physical properties like star-formation histories and metallicities.

Jocelyn: And then they have degraders that intentionally introduce imperfections, such as the LSSTErrorModel or specific spectroscopic degraders to simulate real-world problems.

Subrahmanyan: That systematic degradation is crucial because it lets you stress-test your estimation methods against known types of errors before you deploy them on real data.

Conclusion: Vera: So to wrap up, the main implication of this paper is that they’ve provided a flexible, open-source infrastructure for systematically testing and producing photo-z data products for the Rubin era.

Jocelyn: It moves us away from just trusting a single number and toward using those probability distributions to properly quantify our redshift uncertainties across vast numbers of galaxies.

Subrahmanyan: This infrastructure is important because it addresses the issue of discrepancies between different algorithms, which is something we need to settle before we can use these photo-z products for precise cosmological constraints.

Vera: They are giving us a standardized way to evaluate whether one estimation method performs better than another when faced with realistic noise and errors.

Jocelyn: It’s about making the process of validating photo-z methods more rigorous and transparent for the wider community using these new metrics they’ve introduced.

Subrahmanyan: The RAIL paper gives us a clear framework for how to handle the inherent redshift uncertainties in photometric data, which is a fundamental problem for any future large-scale survey.

Vera: That’s what we have on "Redshift Assessment Infrastructure Layers (RAIL): Rubin-era photometric redshift stress-testing and at-scale production". We’ve seen how they built this testing framework from the ground up.

Jan Luca van den Busch, Eric Charles, Johann Cohen-Tanugi, Alice Crafford, John Franklin Crenshaw, Sylvie Dagoret, Josue De-Santiago, Juan De Vicente, Qianjun Hang, †Benjamin Joachimi†Shahab Joudaki—J. Bryce Kalmbach⁳Arun Kannawadi—Shuang LiangOlivia LynnAlex I. Malz▦Rachel MandelbaumSidney MauLeonel Medina-VarelaGrant MerzIrene MoskowitzDrew OldagJaime Ruiz-ZapateroMubdi RahmanMarkus M. RauSamuel J. SchmidtJennifer ScoraRaphael ShirleyBenjamin StölznerLaura Toribio SanCiprianoLuca TortorelliZiang YanTianqing Zhang, orgs_list_derived_from_affiliations

Ruhr University Bochum Faculty of Physics and Astronomy Astronomical Institute AIRUB · SLAC National Accelerator Laboratory · Kavli Institute for Particle Astrophysics and Cosmology Stanford University · Université Clermont-Auvergne CNRS LPCA · McWilliams Center for Cosmology and Astrophysics Carnegie Mellon University Pittsburgh Department of Physics University of Washington Seattle DIRAC Institute University of Washington Seattle Université Paris-Saclay CNRS ICLab Secihti—Departamento de Física Centro de Investigación y de Estudios Avanzados del I.P.N. · Department of Physics & Astronomy University College London · Department of Astronomy University of Washington · Department of Physics Duke University · Stony Brook University Department of Physics and Astronomy Department of Physics and Astronomy Stony Brook University Max-Planck-Institut für extraterrestrische Physik Giessenbachstrasse 1 Université Ludwig Maximilian University Munich Universitat Sternwarte Fakultät Physik Ludwig-Maximilians-University München · Department of Physics & Astronomy University College London · Department of Astronomy University of Washington Department of Physics Duke University Department of Physics and Astronomy Stony Brook University

astro-ph.IM, astro-ph.CO, astro-ph.GA

Submitted: 2025-05-05

Updated: 2026-10-16

Comments: Accepted by OJAp, 21 pages, 6 figures, 5 tables

DOI: 10.33232/001c.158200

Code: https://github.com/LSSTDESC/RAIL2https:

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: The RAIL team presents an open-source Python library designed for at-scale probabilistic photo-z estimation, which serves as a flexible toolkit for stress-testing and deriving photo-z data products

Key concepts

Forward Modeling Suite: This module generates complex photometry by modeling galaxy spectral energy distributions (SEDs). It incorporates physical properties like star-formation histories and metallicity to create realistic input data for testing photo-z algorithms.
Degraders: These tools introduce tunable imperfections into mock data to simulate real-world errors. Examples include the LSSTErrorModel and SpectroscopicDegraders, which mimic issues like unrecognized blends or redshift incompleteness in observational catalogs.
Distribution-to-Distribution Metrics: These metrics compare the probability distribution of estimated redshifts against the true underlying distribution. They use tests like Cramér von Mises (CvM) and Kolmogorov-Smirnov (KS) to quantify how well an algorithm reproduces the full statistical shape of the redshift data.

Terminology

Redshift Assessment Infrastructure Layers — 2× in this paper
Fidelity — 1× in this paper · explained in 2 episodes
Forward modeling — 1× in this paper · explained in 2 episodes
Legacy Survey of Space and Time — 1× in this paper

Summary

The RAIL team presents an open-source Python library designed for at-scale probabilistic photo-z estimation, which serves as a flexible toolkit for stress-testing and deriving photo-z data products for extragalactic science.

RAIL's Core Functionality

The RAIL software library provides three subpackages offering modular tools for end-to-end stress-testing, including a forward modeling suite to generate realistically complex photometry, a unified API for estimating per-galaxy and ensemble redshift PDFs by an extensible set of algorithms, and built-in metrics of both photo-z PDFs and point estimates (Page 1).
The core structure manages functionalities under a unified workflow, where execution is facilitated in a traceable manner through stages wrapped in pipelines, which are constructed as a directed acyclic graph (DAG) by connecting the inputs and outputs of various Stage objects (Page 3).
Data in RAIL are passed between stages in the form of data handles, which include subclasses like tabular data, PDF data, and models; for instance, a TableHandle wraps catalog-like data, while a qp handle wraps qp ensembles (Page 3).

Creation and Degradation

Mock DESC data are crucial because they must include not only true redshifts and LSST photometry but also true posterior PDFs, p(zpt), which are unavailable for spectroscopically confirmed data sets as well as traditional simulations (Page 4).
The creation module features engines like FSPS, which models galaxy spectral energy distributions (SEDs) by incorporating physical properties such as star-formation histories, metallicities and redshift, and degraders that introduce imperfections, such as the LSSTErrorModel or the SpectroscopicDegraders like InvRedshiftIncompleteness (Page 6).
Degraders include specific tools like the Blending Degrader, which simulates unrecognized blends based on source density by merging fluxes of close objects found via a Friends-of-Friends (FoF) algorithm (Page 8).

Photo-z Estimation Methods

RAIL encompasses all methods that derive redshift information from photometry, implemented through an Informer stage paired with an Estimator, Summarizer, and Classifier, which produces per-galaxy photo z PDFs or point estimates (Page 4).
Various estimation algorithms are available in the rail sklearn package, including CMNN (Color-Matched Nearest Neighbor), DNF (Directional Neighborhood Fitting), and FlexZBoost (Page 9).
Hybrid catalog estimators like Delight combine machine learning and template-fitting methods by implicitly constructing flexible template SEDs directly from the spectroscopic training data, called Delight (Page 10).
Image-based estimators such as DeepDISC utilize instance segmentation models, where the CatEstimator stage outputs a qp.Ensemble containing redshift PDFs along with ancillary information including object RA and Dec (Page 11).

Evaluation Modules

RAIL provides evaluations of photo-z methods through a library of metrics categorized into distribution-to-distribution metrics, distribution-to-point metrics, and point-to-point metrics (Page 13).
Distribution-to-distribution metrics compare the PDFs of the estimated galaxies with the 'true PDF', using measures such as Cramér von Mises (CvM), Kolmogorov Smirnov (KS) test, Root Mean Square Error (RMSE), Kullback Leibler Divergence, and Anderson-Darling (AD) test (Page 14).
Point-to-point metrics compare point estimates against the truth, providing statistics like PointSigmaIQR to approximate Gaussian standard deviation and PointBias to compute the median of PointStatsEz (Page 15).

Performance Benchmarking

A performance study shows that the speed of algorithms varies significantly, with FlexZBoost having a speed of 1600 [galaxies(s × CPU)−1] compared to kNN at 4000 [galaxies(s × CPU)−1] (Page 17).
The paper notes that the goal is not to systematically benchmark all algorithms but rather to compare them under realistic conditions, such as using mock LSST data via the p(z) and n(z) data challenge (Page 18).
Future development plans include incorporating methods like SOMPZ, which uses a 2-tiered SOM to exploit deep field photometry and transfer information consistently (Page 18).

RAIL enables studies that address key challenges identified in DESC’s earlier photo-z experiments, such as discrepancies between algorithms, inadequacies of traditional performance metrics, and the need for probabilistic approaches to model inherent redshift uncertainties. This release represents a critical step toward ensuring that LSST photo-z data products meet the stringent requirements of Rubin’s cosmological and extragalactic science cases while also serving a broader community of researchers with varied scientific goals.

The gist: The RAIL software library is an open-source Python toolkit for at-scale probabilistic photo-z estimation, designed to stress-test and derive photo-z data products for extragalactic science.

How it works

RAIL's core functionality is structured around a tripartite architecture enabling experiments to optimize photo-z data products: creation, estimation, and evaluation (Page 3). This workflow is managed through stages wrapped in pipelines, which are constructed as a directed acyclic graph (DAG) by connecting the inputs and outputs of various Stage objects (Page 3).

The creation subpackage provides tools for generating realistic mock data. This involves engines like FSPS or DSPS to create rest-frame SEDs, followed by degraders that introduce tunable physical imperfections, such as the LSSTErrorModel or SpectroscopicDegraders (Page 6). These mock datasets are important because they must include true posterior PDFs, p(zpt), which are unavailable for spectroscopically confirmed data sets as well as traditional simulations (Page 4).

The estimation subpackage supports a unified API for implementing and comparing diverse algorithms. These algorithms include machine learning methods like CMNN, template-fitting methods such as BPZ, and hybrid catalog estimators like Delight (Page 9). Each estimator is implemented with an Informer stage paired with an Estimator, Summarizer, and Classifier (Page 4).

The evaluation subpackage offers a flexible suite of metrics to quantify performance. These include distribution-to-distribution metrics like Cramér von Mises (CvM) and Kolmogorov Smirnov (KS) test, as well as point-to-point metrics such as PointSigmaIQR to approximate Gaussian standard deviation (Page 13).

Key Tools and Components

The RAIL ecosystem relies on several key dependencies:

** tables io abstracts the handling of catalog data from RAIL and other DESC software, allowing such conversions to be performed automatically without the user having to manually preprocess data (Page 21). **

**/qp is a Python package for handling univariate PDFs, encapsulating all PDFs over redshift by qp.Ensemble objects (Page 21). **

The system uses various stages and tools, such as the HyperbolicMagnitudes in photometry tools to convert classical magnitudes to hyperbolic magnitudes (Page 21), and the TableConverter in table tools to convert tables from one format to another (Page 21).

Conclusion

RAIL provides many recent photo-z algorithms, including machine learning, template fitting, hybrid, and image-based algorithms for per-galaxy photo z estimation. RAIL provides a common infrastructure for training models and estimating redshift PDFs which are parameterised by qp (Page 17). RAIL enables studies that address key challenges identified in DESC’s earlier photo-z experiments, such as discrepancies between algorithms, inadequacies of traditional performance metrics, and the need for probabilistic approaches to model inherent redshift uncertainties.

RAIL is open-source, modular, and extensible with intended usage throughout and beyond the Rubin ecosystem. The RAIL team is continuing to add more metrics into the codebase, especially those directly connected to specific science cases (Page 15). Future development will focus on expanding the range of supported algorithms, incorporating feedback from early users, and addressing emerging challenges in photo-z systematics (Page 18).

RAIL is open-source, modular, and extensible with intended usage throughout and beyond the Rubin ecosystem.

Improvements for AI systems

Bold Header: Modular End-to-End Photo-z Pipelines

The improved AI system can now execute a creation, estimation, and evaluation workflow by chaining modules like FSPS for SED generation with PZFlow for estimation and then applying metrics from the evaluation subpackage to stress-test the entire pipeline.

Bold Header: Robust Mock Data Generation

The system can generate mock DESC data that includes not only true redshifts and LSST photometry (i.e., fluxes in the six LSST bands) but also true posterior PDFs, p(zp), which are unavailable for spectroscopically confirmed data sets as well as traditional simulations.

Bold Header: Automated Systematic Uncertainty Stress-Testing

The AI can perform stress-testing of photo-z methodologies by applying a sequence of degraders, such as the LSSTErrorModel, InvRedshiftIncompleteness selector, and lineConfusion degrader, to the mock catalog before running an estimator.

Bold Header: Comprehensive Metric Evaluation

The system can compare algorithms using a suite of metrics, including Distribution-to-Distribution Metrics like Cramér-von Mises (CvM), Kolmogorov-Smirnov (KS) test, Root Mean Square Error (RMSE), and Kullback-Leibler Divergence.

Bold Header: Probabilistic Point Estimate Validation

The AI can evaluate the performance of point estimates by computing Conditional density loss (CDELoss) and Probability Integral Transformation (PIT), ensuring that the estimated PDFs are consistent with the true redshift PDF through a PIT-QQ diagram comparison against a uniform distribution.

Bold Header: Ensemble Redshift Distribution Summarization

The system can generate ensemble redshift distributions, including the NaiveStackSummarizer and sophisticated methods like NZDir, to provide summaries of the sample's overall redshift distribution, which can be used as input for cosmological constraints.

Bold Header: Joint Constraint Generation via Clustering

The AI can leverage YetAnotherWizz (YAW) to estimate the ensemble redshift distribution based on spatial clustering, providing a powerful, independent estimate that can be combined with photometric redshift estimation methods to yield a joint constraint on n(z).

Abstract

Virtually all extragalactic use cases of the Vera C. Rubin Observatory's Legacy Survey of Space and Time (LSST) require the use of galaxy redshift information, yet the vast majority of its sample of tens of billions of galaxies will lack high-fidelity spectroscopic measurements thereof, instead relying on photometric redshifts (photo- z) subject to systematic imprecision and inaccuracy best encapsulated by photo- z probability density functions (PDFs). We present the version 1 release of Redshift Assessment Infrastructure Layers (RAIL), an open source Python library for at-scale probabilistic photo- z estimation, initiated by the LSST Dark Energy Science Collaboration (DESC) with contributions from the LSST Interdisciplinary Network for Collaboration and Computing (LINCC) Frameworks team. RAIL's three subpackages provide modular tools for end-to-end stress-testing, including a forward modeling suite to generate realistically complex photometry, a unified API for estimating per-galaxy and ensemble redshift PDFs by an extensible set of algorithms, and built-in metrics of both photo- z PDFs and point estimates. RAIL serves as a flexible toolkit enabling the derivation and optimization of photo- z data products at scale for a variety of science goals and is not specific to LSST data. We thus describe to the extragalactic science community, including and beyond Rubin the design and functionality of the RAIL software library so that any researcher may have access to its wide array of photo- z characterization and assessment tools.