Neptuna: A Comprehensive Machine Learning Framework for Benchmarking Complex Multiphase Flows

arXiv:2607.22280 · physics.flu-dyn, cs.AI · Submitted 2026-07-24 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Neptuna: A Comprehensive Machine Learning Framework for Benchmarking Complex Multiphase Flows".

Jane: Here's the gist of the paper. Detailed Research Summary of "Neptuna: A Comprehensive Machine Learning Framework for Benchmarking Complex Multiphase Flows" This research introduces Neptuna, a comprehensive machine learning framework specifically designed to establish a rigorous, large-scale benchmark for training and evaluating surrogate models for complex, shock-driven compressible multiphase flows.

Tom: The work addresses the critical need for robust data-driven surrogates in fields where high-fidelity simulations are computationally prohibitive.

Jane: Core Dataset and Scope: The foundation of this work is a massive 2.4 Terabyte (TB) high-fidelity benchmark dataset.

Tom: This dataset is meticulously curated to cover a wide range of physical phenomena, specifically focusing on shock-driven compressible multiphase flows, including intricate processes such as bubble collapse and droplet breakup, across both 2D and 3D geometries.

Paper summary: Tom: So, we’re wrapping up this look at Neptuna, which is this new framework for testing machine learning models on really messy fluid flows.

Jane: Basically, they took a ton of complex data—three terabytes of it—and built a system to see if different AI approaches can handle the shocks and mixing in those flows.

Lu: They weren’t just looking at standard neural networks; they included convolutional models, spectral methods, and even pre-trained physics foundation models for comparison <ref:2607.22280#pg3>

Meng: And the key part is that they looked beyond just simple error scores like mean squared error; they used composite losses that look at interfaces and structure <ref:2607.22280#pg3>

Tom: Yeah, those composite losses were pretty important because it showed that no single model was best on everything, but combining them made a real difference in keeping the flow shapes correct <ref:2607.22280#pg3>

Jane: And they found that an adaptive strategy called SoftAdapt worked really well to improve things with almost no extra training work needed compared to just using basic error training <ref:2607.22280#pg3>

Lu: What’s interesting is how they used these metrics—pointwise, spectral, and structure-aware—to give a full picture of the solution quality <ref:2607.22280#pg3>

Meng: From an engineering standpoint, it shows that for real-world applications where you need stability at scale, those composite losses and adaptive methods are actually pretty practical <ref:2607.22280#pg3>

Tom: The authors call this Neptuna a comprehensive framework for benchmarking these complex multiphase flows.

Jane: It sets a new standard for how we evaluate AI surrogates when they’re dealing with shocks and interfaces all at once <ref:2607.22280#pg3>

Lu: It really highlights the need to test models against datasets that are genuinely challenging, like those involving shock-driven bubble collapse and droplet breakup <ref:2607.22280#pg3>

Meng: The implication is that if you want to build a reliable AI for things like underwater acoustics or material simulations, you need this kind of rigorous testing upfront <ref:2607.22280#pg2>

Tom: So, Neptuna isn't just about one model; it’s about building a better way to test the whole machine learning pipeline for these difficult physics problems <ref:2607.22280#pg3>

Jane: It gives us a concrete set of tools to judge whether an AI surrogate is actually capturing the physics of these violent fluid interactions correctly <ref:2607.22280#pg3>

Conclusion: Tom: So, we're wrapping up this look at Neptuna, which is this new framework for testing machine learning models on really messy fluid flows that involve shocks and mixing.

Jane: Basically, they took a massive test set—three terabytes of data in 2D and three dee—and built a system to see if different AI approaches can handle those violent shock-driven flows <ref:2607.22280#pg1>.

Lu: They tested several model types: convolutional networks, spectral methods, transformer-based models like FFNO, and even pre-trained physics foundation models for comparison.

Meng: And they didn't just use simple error scores; they used composite losses that look at derivatives and interface structure.

Tom: Yeah, those composite losses were important because no single model won every time across all the metrics—pointwise, spectral, and structure-focused ones. But combining them made a real difference in keeping the flow shapes correct.

Jane: And they found that an adaptive strategy called SoftAdapt worked really well to improve things with almost no extra training work compared to just using mean squared error alone.

Lu: What's interesting is how they used these metrics—pointwise, spectral, and structure-aware—to give a full picture of the solution quality.

Meng: From an engineering standpoint, if this framework works, it means we can finally start comparing different AI approaches fairly against a really hard problem that’s actually relevant to simulation.

Tom: The implication is that for applications like bubble curtains or shockwave lithotripsy, where you need accurate predictions for complex multiphase interactions, this kind of rigorous testing is what we need to build reliable surrogates.

Jane: So, Neptuna gives the community a standardized way to say: "This model works well at this level of complexity for shock-driven flows." It’s about making sure the AI solutions are physically meaningful and structurally sound.

Lu: And they also pointed out a limitation, which is pretty honest. They said there’s still a gap in how well current models capture those very fine-scale details, like the small secondary droplets that break off during droplet breakup in the SIE regime.

Meng: That missing detail is what I worry about for production. If the AI can’t handle those tiny secondary effects, it might fail in a real-world scenario where you need every little bit of detail to be correct.

Tom: So, while they did a great job setting up this framework and showing that composite losses are good, the next step is clearly focusing on fixing that fine-scale fidelity issue they pointed out.

Jane: And for the world listening right now, it means we have a better way to train AI to understand how fluids behave when things are violently interacting with shocks, which is relevant everywhere from underwater engineering to materials science.

Harish Ramachandran, Björn Kimpel, Thomas Paula, Josef Winter, Steffen Schmidt, Nikolaus Adams

Technical University of Munich

physics.flu-dyn, cs.AI

Submitted: 2026-07-24

Updated: 2026-10-18

Comments: Accepted at NeurIPS-2026 (Evaluations and Datasets Track)

Project page: https://tumaer.github.io/Neptuna

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 86/100

The gist: Detailed Research Summary of "Neptuna: A Comprehensive Machine Learning Framework for Benchmarking Complex Multiphase Flows" This research introduces Neptuna, a comprehensive machine learning

Key concepts

Neptuna: A comprehensive machine learning framework designed to create a rigorous benchmark for training and evaluating surrogate models for complex, shock-driven compressible multiphase flows. It combines a large dataset with diverse model testing and advanced loss functions.
Composite Losses: Instead of just using standard error (MSE), this involves combining MSE with specialized terms like Sobolev norms or interface-aware losses. This forces the machine learning model to learn not only the general flow but also critical physical details like sharp interfaces and gradients.
SoftAdapt: An adaptive loss balancing strategy used during training. It automatically adjusts the weighting between different loss components (like MSE and structural terms) to optimize performance without requiring manual tuning, proving highly effective for improving surrogate model consistency.

Terminology

Benchmarking — 2× in this paper · explained in 4 episodes
Fidelity — 2× in this paper · explained in 2 episodes
Adaptive Weighting — 1× in this paper
Benchmark — 1× in this paper · explained in 2 episodes
Foundation Model — 1× in this paper · explained in 3 episodes
Foundation Models — 1× in this paper · explained in 7 episodes
Surrogate Model — 1× in this paper · explained in 3 episodes
Transformer — 1× in this paper · explained in 2 episodes

Summary

Detailed Research Summary of Neptuna: A Comprehensive Machine Learning Framework for Benchmarking Complex Multiphase Flows

This research introduces Neptuna, a comprehensive machine learning framework specifically designed to establish a rigorous, large-scale benchmark for training and evaluating surrogate models for complex, shock-driven compressible multiphase flows. The work addresses the critical need for robust data-driven surrogates in fields where high-fidelity simulations are computationally prohibitive.

Core Dataset and Scope:

The foundation of this work is a massive 2.4 Terabyte (TB) high-fidelity benchmark dataset. This dataset is meticulously curated to cover a wide range of physical phenomena, specifically focusing on shock-driven compressible multiphase flows, including intricate processes such as bubble collapse and droplet breakup, across both 2D and 3D geometries.

Benchmarking Diverse Surrogate Models:

Neptuna's primary contribution is the systematic evaluation of a diverse array of surrogate model architectures against this challenging dataset. The study benchmarks several state-of-the-art families, including:

Convolutional Networks (e.g., ConvNeXt)
Spectral Methods
Transformer-based Models (e.g., FFNO)
Pre-trained Physics-Informed Neural Network (PDE Foundation) models.

The evaluation framework is designed to be holistic, moving far beyond simple pointwise error metrics.

Advanced Loss Functions and Adaptive Training Strategies:

A key innovation lies in the development of physics- and structure-aware composite loss functions. Instead of relying solely on standard Mean Squared Error (MSE), the authors investigate combining MSE with specialized terms that target specific physical aspects:

H1-seminorm: To enforce fidelity concerning spatial gradients.
Interface-focused Root Mean Square Error (IRMSE): Crucial for accurately capturing sharp interfaces.
Structural Similarity Index (SSIM): To ensure the learned solutions maintain the correct structural integrity of the flow features.

To optimize training stability and performance across various baselines, adaptive loss balancing strategies, specifically SoftAdapt, are employed. Results consistently demonstrate that SoftAdapt provides the most reliable improvements, achieving superior results with almost no computational overhead compared to standard MSE-only training schemes. GradNorm is also explored as an alternative adaptive weighting strategy.

Comprehensive Evaluation Metrics:

The evaluation framework is exceptionally thorough, extending beyond pointwise errors to capture multiple facets of the solution quality:

Spectral Metrics: Such as Multilevel Wavelet Loss (MLW), assessing fidelity in the frequency domain.
Feature-Focused Metrics: Including IRMSE and Shock-Focused RMSE, specifically designed to measure preservation of critical features like interfaces and shock waves.
Structure-Aware Measures: PCC (Pearson Correlation Coefficient) and SSIM, quantifying how well the learned structure matches the ground truth structure.
Physics-Informed Metrics: Tracking integral quantities of interest (e.g., mass, momentum) using normalized Root Mean Square Error (nRMSE), ensuring that the surrogate models respect underlying physical laws.

The paper systematically tracks these metrics across different training timesteps and model architectures for both bubble collapse and droplet breakup scenarios on the 3D-SABW (SSOOSS) and 3D-SDBA (SSOOSS) test datasets.

Key Findings and Contributions:

Composite Losses are Superior: The integration of physics-aware composite losses significantly enhances interface preservation and spectral fidelity, proving that no single model excels across all metrics.
SoftAdapt's Efficacy: SoftAdapt emerges as the most consistent method for improving surrogate performance with minimal training complexity.
Holistic View: The framework successfully demonstrates the necessity of incorporating complex, high-fidelity datasets and a multi-faceted evaluation suite (pointwise, spectral, structural, physics-informed) into the SciML community to build truly robust data-driven models.

Limitations and Future Directions:

Despite its comprehensive nature, the authors acknowledge significant limitations. Specifically:

Fine-Scale Detail Capture: There remains a substantial gap in the ability of current surrogate models to accurately recover fine-scale details, such as the small secondary droplets that detach from primary droplets during droplet breakup (a defining characteristic of the SIE regime).
Boundary Condition Conditioning: Future work is strongly suggested to focus on incorporating detailed information about boundary conditions directly into the training process, which is anticipated to improve the generalizability of surrogate models across varied simulation setups.

In summary, Neptuna establishes a gold standard for benchmarking complex multiphase flow surrogates by combining massive, diverse datasets with sophisticated composite loss functions and a multi-dimensional evaluation protocol.

Improvements for AI systems

Adaptive loss weighting for surrogate models can significantly improve interface preservation and spectral fidelity in complex flows, as shown by composite losses combining MSE with Sobolev, interface-aware, and structure-aware terms.
A surrogate model trained with a composite loss using SoftAdapt adaptive loss balancing strategy can achieve substantial improvements in the Interface-focused Root Mean Square Error (IRMSE) for shock-droplet interaction datasets.
The system can be evaluated using a comprehensive set of metrics including pointwise, spectral, feature-focused, structure-aware, and physics-informed metrics to provide a multi-faceted assessment of model performance.
A surrogate model can be trained to accurately predict the evolution of mass, momentum (x and y components), kinetic energy, and enstrophy by minimizing integral quantities of interest (iqRMSE).
The system can perform conservative downsampling when reducing data resolution to ensure conservation in total mass, momentum, and energy fields during the creation of smaller training datasets.
A surrogate model trained with composite losses can show superior performance in capturing structural coherence, as evidenced by higher Structural Similarity Index (SSIM) values during inference.
The system can be fine-tuned from pre-trained foundation models like Poseidon or DPOT to achieve high accuracy on specific flow regimes, such as shock-induced air bubble collapse in water.
The system can dynamically condition the neural operator architectures using learned scale and shift terms derived from conditioning vectors via AdaNorm, improving generalization across different simulation setups.

Abstract

Compressible multiphase flows involving shocks and material interfaces arise in applications such as bubble collapse and droplet breakup, where strong nonlinear interactions produce complex interface deformation, mixing, and multiscale dynamics. Developing reliable machine learning surrogates for these flows remains challenging due to the simultaneous presence of compressibility, sharp discontinuities, and multiphase effects. In this work, we introduce the first large-scale benchmark specifically designed for shock-driven compressible multiphase flows, comprising 3 TB of high-fidelity 2D and 3D datasets featuring shock-induced bubble collapse and droplet breakup. We evaluate diverse surrogate model families on our benchmarking framework: Neptuna, including convolutional, spectral, transformer-based, and pre-trained PDE foundation models. Beyond standard MSE training, we investigate composite losses that combine MSE with Sobolev, interface-aware, and structure-aware terms, along with adaptive loss balancing using SoftAdapt and GradNorm. Evaluation includes pointwise, spectral, feature-focused, structural, and physics-informed metrics. Results show that no single model performs best across all datasets and metrics, while composite losses significantly improve interface preservation and spectral fidelity. Among adaptive weighting strategies, SoftAdapt provides the most consistent improvements with almost no overhead compared to MSE-only training. The benchmarking repository and the dataset are made available at