Blending data and physics for reduced-order modeling of systems with spatiotemporal chaotic dynamics

The gist: This work develops a hybrid reduced-order model (ROM), informed by both data and physics, for evolving spatiotemporal chaotic dynamics on an invariant manifold whose coordinates are found using an autoencoder.

Hybrid ROM Framework

The approach combines the purely data-driven method Data-driven manifold dynamics (DManD) with the physics-based Manifold Galerkin (MG) method to create a hybrid ROM framework called physics-informed DManD (PIDManD) The hybrid ROM is built by combining MG-N with DManD in two ways, which are illustrated in the bottom row of Figure 1(c). The first method, PI-DManD-c, adds a correction term learned by a NODE to the MG-N vector field in order to account for numerical and inherent errors in the FOM. The second hybrid ROM is PI-DManD-B, which uses the MG-N model as a Bayesian prior for training a NODE model.

Data-driven Manifold Dynamics (DManD)

The first step in building a ROM with the present approach is to use an undercomplete autoencoder (AE) to find a low-dimensional, high-fidelity latent representation of u. An AE consists of two parts: the encoder E nonlinearly maps u to the latent space h ∈ R dh, where dh < du, and the decoder D performs the reverse mapping ˜u = D(h) = D(E(u)). With the AE in hand, the DManD approach to reduced-order modeling learns a dynamical system in the latent space: dh/dt = g(h), where g is the true vector field in the manifold coordinates. This vector field g is approximated as a neural network gd using the NODE method.

Manifold Galerkin (MG)

The Manifold Galerkin (MG) method directly expresses the vector field fFOM in the manifold coordinates h. This is achieved by substituting u = D(h; θD) into the FOM and using automatic differentiation to compute dh/dt = gm(h; θE, θD, µFOM), where gm = JE (D(h; θD); θE) fFOM(D(h; θD); µFOM). Although MG in Equation 13 has the appearance of a ROM since it provides the vector field in the latent space h, it is in fact even slower to solve than the FOM because it must first decode h to u and evaluate a Jacobian at each timestep. This method is described as a physics-based method (but not a ROM) that projects the physics of the FOM onto the manifold coordinates found with a data-driven autoencoder.

Physics-informed DManD (PI-DManD)

The PI-DManD methods incorporate physics into DManD to create two separate methods. The first method, PI-DManD-c, adds a correction term gc(h) to the vector field from MG-N to get the final, reduced-order dynamical model in the latent coordinates: dh/dt = gmn(h; θgmn) + gc(h; θgc). The second PI-DManD approach, PI-DManD-B, uses the MG-N model as a prior for training the NODE model. The loss used by PI-DManD-B can be viewed as being motivated from a Bayesian framework.

Results and Robustness

The hybrid ROMs show substantial improvements in predictive capability relative to an approach using only data. In the case of abundant data and exact FOM, PI-DManD-B (and PI-DManD-c) accurately tracked the true solution for substantially longer, with the ensemble-averaged relative short-time tracking error being 0.05 for the hybrid ROMs compared to 0.1 for DManD. The results also demonstrate robustness to scarce data and FOM errors, as PI-DManD-B still provided sizable improvements over DManD in short-time tracking when using an erroneous parameter (L = 20) in the FOM.

Computational Cost

While the total time for training either hybrid methods is greater than that of DManD, they still take a similar amount of time to evaluate/test and have the added benefit of substantially better short-time tracking of the true solution. MG is clearly not a ROM because it must decode h, solve the FOM, and evaluate a Jacobian at each timestep. The hybrid ROMs are non-intrusive if fFOM,i data are available but require a preprocessing step of calculating fFOM,i from ui for all i if the dataset comes from experiments.

Long-time Statistics

For long-time statistics, the Earth mover’s distance (EMD) between pdfs of a model prediction and the ground truth trajectory is calculated. In the case of abundant data and exact FOM, all five methods performed equally well in capturing the long-time statistics, as their EMD values were similar to the baseline EMD. When given an erroneous parameter value for the FOM, MG and MG-N evolved the β = -2 dataset with a completely incorrect vector field, leading to poor reconstruction of long-time statistics. The two hybrid methods captured the long-time statistics about as well as DManD in these erroneous physics cases.

Complex Ginzburg–Landau Equation

The study also applied PI-DManD for prediction of dynamics of the one-dimensional complex Ginzburg–Landau equation (CGLE). The dimension of the invariant manifold for this CGLE system was estimated to be dM = 40 using an implicit rank-minimizing autoencoder. In abundant data and exact FOM for the CGLE, MG showed dramatically lower short-time tracking error compared to DManD. The superior predictive performance of MG for the CGLE comes at a large cost, as MG still takes almost two orders of magnitude longer to evolve a trajectory than other methods.

Summary Table

The summary table shows that PI-DManD-B provided % reduction in relative trajectory error at t = τL ranging from 43%–64% under erroneous physics conditions. The results demonstrate that including knowledge of the physics (even if inaccurate) helps in creating hybrid ROMs (PI-DManD-B especially) that perform better than solely data-driven ROMs (DManD) in short-time predictions.

Conclusion

In this work, we found that including physics-based prior information substantially improves the predictive capabilities of data-driven ROMs applied to systems with spatiotemporal chaotic dynamics, even if the physics-based information is not completely correct. The hybrid ROMs PI-DManD-c and PI-DManD-B are robust to scarce data and FOM errors, demonstrating that including knowledge of the physics (even if inaccurate) helps in creating hybrid ROMs (PI-DManD-B especially) that perform better than solely data-driven ROMs (DManD) in short-time predictions. Future directions include testing robustness of our hybrid ROMs on complex fluid and flow systems for which only approximate first-principles models are available.

References

[1] P. Holmes, J. L. Lumley, G. Berkooz, and C. W.

1. Accuracy improvement for Reduced Order Models (ROMs). The hybrid ROM approach, specifically PI-DManD-B, yields substantially improved time-series predictions relative to a data-only approach across scenarios of abundant data, scarce data, and even an incorrect full order model (FOM).

2. Robustness against FOM inaccuracies. The PI-DManD-B method demonstrates that even if the FOM of the physics based prior is highly erroneous, it can still substantially outperform the purely data driven DManD ROM in short-time tracking.

3. Handling of data scarcity and sampling rates. The hybrid ROMs, particularly PI-DManD-B, maintain equal or lower error than DManD across three subcases of scarce data training, showing robustness when dealing with shorter time series and coarser sampling rates compared to the baseline DManD model.

4. Long-time statistical accuracy preservation. For both the Kuramoto–Sivashinsky equation and the complex Ginzburg–Landau equation, all five methods capture long-time statistics equally well, indicated by similar Earth mover’s distance (EMD) values between model predictions and ground truth trajectories.

5. Efficient inference for high-fidelity models. While the Manifold Galerkin (MG) method is not a ROM because it is even slower to solve than the FOM, using a neural network approximation (MG-N) allows the hybrid models to achieve comparable test time to DManD, enabling faster evaluation of initial conditions for 100 Lyapunov times.

6. Error correction capability in predictive dynamics. The PI-DManD-c method introduces a corrective neural ODE term gc, which is used to fix numerical and FOM approximation errors, leading to better short-time tracking than the MG-N model alone when the FOM is inaccurate.

While data-driven techniques are powerful tools for reduced-order modeling of systems with chaotic dynamics, great potential remains for leveraging known physics (i.e. a full-order model (FOM)) to improve predictive capability. We develop a hybrid reduced order model (ROM), informed by both data and FOM, for evolving spatiotemporal chaotic dynamics on an invariant manifold whose coordinates are found using an autoencoder. This approach projects the vector field of the FOM onto the invariant manifold; then, this physics-derived vector field is either corrected using dynamic data, or used as a Bayesian prior that is updated with data. In both cases, the neural ordinary differential equation approach is used. We consider simulated data from the Kuramoto-Sivashinsky and complex Ginzburg-Landau equations. Relative to the data-only approach, for scenarios of abundant data, scarce data, and even an incorrect FOM (i.e. erroneous parameter values), the hybrid approach yields substantially improved time-series predictions.