A Robust Estimator for the Lyman- alpha Forest 1D Power Spectrum at High Redshift

The gist The likelihood-based estimator recovers the input power spectrum across tested scales well within expected error and remains robust to observational effects

Introduction and Motivation

Measurements of the Lyman-α (Lyα) forest flux power spectrum P(k) are sensitive to a range of astrophysical and cosmological effects, including the thermal state of and large-scale fluctuations in the intergalactic medium and the nature of dark matter. Measuring P(k) becomes increasingly challenging at high redshift, where fewer quasar spectra are available and higher Lyα optical depths result in lower transmitted flux. The enhanced scatter in the Lyα forest opacity at z > 5.5 cannot be explained by density fluctuations alone. This motivates models that invoke spatial fluctuations in the ultraviolet background (UVB) and the IGM temperature. Robustly measuring P(k) across a range of scales is therefore important for extracting the physical information encoded in the Lyα forest.

Mock Data Generation

The authors generate mock spectra to test their estimator using three classes of data 2.1 Simulations use a hydrodynamical simulation performed with the Nyx code [64] which incorporates updated physical rates described by [65]. They use boxes of fluctuating ΓUVB values generated with the semi-numerical method of [19] to model spatial fluctuations, using λmfp = 5, 10, 30, and 150 cMpc. The analytical mocks generate Gaussian random field realizations from a known input power spectrum P1D(k) defined by a power law with an exponential cutoff.

Methods for Power Spectrum Estimation

The paper compares two main approaches to estimating P(k): the standard Fourier transform and a likelihood-based estimator. The Fourier transform method uses the window function WR(k, σR, ∆v) to correct for instrumental resolution and pixel size effects, resulting in P(k) = Praw(k) − Pnoise(k)W2/R (k, σR, ∆v). The likelihood-based method maximizes the likelihood of observed flux fluctuations given a model for their covariance matrix Cij = ⟨δF iδF j⟩ = ⟨sisj⟩ + C N ij. This covariance matrix is constructed using the signal covariance C S ij which is related to the power spectrum via Fourier transforms.

Effective Wavenumber and Bandpower

Since the likelihood estimator constrains bandpowers Pl through their contribution to the covariance matrix, they define an effective wavenumber keff,l = Tr C−1 fid KlC−1 fid Kkl. Similarly, they define a projected effective bandpower Peff,l = Tr C−1 fid KlC−1 fid KPl. These quantities are used to characterize the scales probed by each bin and provide a representative power that accounts for covariance weighting.

Validation Results

The validation tests were performed using three classes of mock data: analytic P(k) mocks, simple forward-modeled simulation mocks, and XQR-30-like forward-modeled simulation mocks. For the analytic mocks, both the likelihood optimization and FFT methods recovered the input P(k) well. For the simple forward-modeled simulation mocks, the 1σ regions of the residuals were consistent with zero across the full k range. Furthermore, realistic XQR-30 masks did not introduce a measurable bias in the recovered P(k), though they increased scatter.

Conclusion

The likelihood-based method accounts for window functions in the transmitted flux field without introducing significant bias into the optimized P(k) measurement. The work validates this method across three increasingly realistic mock-data scenarios where the mock measurements are consistent with the reference model values within expected uncertainties. The likelihood based method is shown to be a robust estimator for measuring the Lyα forest P(k) at z ≥ 5.

How it works

1. The likelihood-based estimator constructs the pixel-level covariance matrix of the transmitted flux field using precomputed k-space integrals that account for instrumental resolution and pixelization, enabling direct maximum-likelihood inference of P(k).

2. It tests the estimator using mock Lyα forest spectra at z ≥ 5, including analytic Gaussian realizations generated from known power spectra and forward-modeled simulation spectra incorporating observational effects relevant to high-redshift quasar surveys.

3. The method constructs the covariance matrix Cij = ⟨δF iδF j⟩ = ⟨sisj⟩ + C N ij, where C S ij describes the correlation between transmitted Lyα forest flux fluctuations in pixels i and j separated by a velocity lag ∆v(i − j).

4. The signal covariance is expressed as C S ij (P) = X Nl l Pl Z kl kl−1 2 cos[k∆v(i − j)] WR(k, Ri, ∆v) × WR(k, Rj, ∆v) dk.

5. The optimal measurement is determined by maximizing the combined likelihood L(P(k)) = N Ysp sp=1 Lsp(P(k)).

6. Effective wavenumbers keff,l and effective bandpowers Peff,l are defined using Fisher-weighted averages to characterize the scales probed by each bin and.

Optimization Bins

The choice of k-bins affects the optimized P(k) measurements, with tests showing that the lowest-k bin must extend below the fundamental mode to avoid biased large-scale measurements. Subdividing this region further produces poorly constrained band powers.

Higher-resolution Analytic Mocks

Repeating the analysis using higher-resolution skewers with ∆v = 4 km/s shows that the optimization residuals remain consistent with zero through the highest-k bin considered. This supports the interpretation that the high-k deviation in the optimization residuals arises from neglecting the window function in the analytic model.

Acknowledgments

MW acknowledges useful conversations with Paul Martini and the CCAPP Cosmology group. JFH acknowledges support from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation program (grant agreement No 885301). This research also used resources of the National Energy Research Scientific Computing Center (NERSC), a U.S. Department of Energy Office of Science User Facility located at Lawrence Berkeley National Laboratory, operated under Contract No. DE-AC02-05CH11231. Generative AI tools (OpenAI’s ChatGPT and Microsoft Copilot) were used to assist with language editing and manuscript revision, the development of plotting scripts used to produce the figures, and the development of the analytic mock function software. MW independently validated the analytic mock function software.

References

[1] Planck Collaboration, N. Aghanim, Y. Akrami, M. Ashdown, J. Aumont, C. Baccigalupi et al., Planck 2018 results. VI. Cosmological parameters, A&A 641 (2020) A6 [1807.06209]

[2] J.E. Gunn and B.A. Peterson, On the Density of Neutral Hydrogen in Intergalactic Space., ApJ 142 (1965) 1633

[3] R. Lynds, The Absorption-Line Spectrum of 4c 05.34, ApJ 164 (1971) L73

[4] A.A. Meiksin, The physics of the intergalactic medium, Reviews of Modern Physics 81 (2009) 1405 [0711.3358]

[5] M. McQuinn, The Evolution of the Intergalactic Medium, ARA&A 54 (2016) 313 [1512.00086]

[6] G.D. Becker, J.S. Bolton, P. Madau, M. Pettini, E.V Ryan-Weber and BP Venemans, Evidence of patchy hydrogen reionization from an extreme Lyα trough below redshift six, MNRAS 447 (2015) 3402 [1407.4850]

[7] G.D. Becker, F.B Davies, S.R Furlanetto, M.A Malkan, E Boera and C Douglass, Evidence for Large-scale Fluctuations in the Metagalactic Ionizing Background Near Redshift Six, ApJ 863 (2018) 92 [1803.08932]

[8] A.-C Eilers, F.B Davies and J.

1. Robust Power Spectrum Estimation in High Redshift IGM simulations: The improved system can perform maximum-likelihood inference of P(k) directly from transmitted flux field data by constructing a pixel-level covariance matrix using precomputed k-space integrals that account for instrumental resolution and pixelization.

2. Modeling Large Scale Fluctuations from UVB Variations: The system can quantify large scale fluctuations in the Ly alpha forest power spectrum by incorporating semi-numerical prescriptions for ultraviolet background fluctuations, where The UVB fluctuations, characterized by the ionizing photon mean free path, λmfp, imprint themselves on the Lyα forest transmitted flux through their effect on the Lyα optical depth.

3. Robust Comparison of Estimation Methods: The improved system can rigorously compare different P(k) estimation techniques (FFT versus likelihood optimization) against known analytic models and realistic mock data by utilizing effective wavenumber (keff) and effective bandpower (Peff) definitions to account for covariance weighting.

4. Realistic Observational Effect Modeling: The system can generate mock spectra incorporating observational effects such as finite spectral resolution, pixel sampling, noise, and missing data relevant to high-redshift quasar surveys, allowing it to test the estimator's robustness against these complexities.

5. Accurate Uncertainty Quantification: The system can estimate expected measurement uncertainties for datasets of varying sizes by computing covariance matrices using a large ensemble of mock datasets (e.g., Σ = 1/Nmocks Σ i=1 (P(k)FFT,i − P(k)FFT,model)).