Fourier neural operator for real-time simulation of 3D dynamic urban microclimate

arXiv:2308.03985 · cs.LG, cs.CE, cs.NA, math.NA, physics.ao-ph, physics.flu-dyn · Submitted 2023-08-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Fourier neural operator for real-time simulation of 3D dynamic urban microclimate".

Jane: The gist The Fourier Neural Operator (FNO) model has been applied for real-time three-dimensional (3D) urban wind field simulation, enabling real-time simulation of 3D dynamic urban microclimate <ref:2308.03985#pg2>.

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper, "Fourier neural operator for real-time simulation of three dee dynamic urban microclimate <ref:2308.03985#pg1,Fourier neural operator for real-time simulation of 3D dynamic urban microclimate>." It sounds pretty technical, but basically, it's about using this specific deep learning model to simulate how wind moves in a city in three dimensions and in real time.

Jane: Right. So when you think about urban microclimates—the way the air feels different right next to buildings—it’s complicated because the physics are non-linear, and traditional computer fluid dynamics simulations can take too long for what we need, which is real-time results.

Lu: The key idea here is that the Fourier neural operator, or FNO, is a machine learning tool that has been shown to be really good at solving partial differential equations over time. It’s like it learns the underlying physics directly from data instead of needing you to write down every single equation by hand.

Meng: So what’s the actual application here? Is it just theoretical work, or is this something that actually helps with city planning or energy efficiency?

Tom: It's focused on real-time simulation, which is crucial because if you want to model dynamic urban microclimates—things like how wind changes instantly—you need speed. This paper points out that this FNO model can accelerate solving these equations quite a bit.

Jane: And it uses data generated from computational fluid dynamics simulations of an urban area, specifically using something called CityFFD60 as the source for training data on page two of the paper <ref:2308.03985#pg1>. So they are learning from what CFD predicts about how wind behaves in a city.

Lalam: From my perspective, this is about taking complex physics problems and finding a way to model them much faster than traditional numerical solvers. It’s about using an AI approach to get dynamic urban microclimate simulations in real time.

The paper's summary: Tom: So, what does the paper actually say about how they did this? They use the FNO model to predict the absolute velocity magnitude of three dee wind fields <ref:2308.03985#pg1>. This means they’re not just looking at one dimension, but all three—the x, y, and z directions—and tracking how that magnitude changes over time.

Jane: I see. So for someone listening who isn't a machine learning expert, imagine you have a map of a city with wind speed numbers changing constantly; this model tries to predict those numbers instantly based on what it saw during training.

Lu: The methodology involves generating data from CFD simulations, and then they train the FNO model using this data, dividing the samples by input and output time steps. To make sure their setup fits on a single Tesla A100 GPU, they have to downsample the original three dee wind fields using a third-order spline interpolation <ref:2308.03985#pg1>.

Meng: That sounds like a lot of preprocessing work before you even start training. What’s the input tensor shape they end up using? We need to know how much data this thing is actually processing.

Tom: The input tensor, which we call a(x), has a shape of two hundred by two hundred by one hundred fifty by five. That means the first three dimensions describe the grid mesh in space—the x, y, and z directions—and the last dimension of five is the number of sequential time steps they are looking at.

Jane: And that’s what lets you see it as a sequence of snapshots over time, which is perfect for dynamic systems like wind fields. So they are training this operator to map inputs to outputs across these spatial and temporal dimensions.

Lalam: Basically, the core idea is using the FNO architecture, which involves lifting the input data into a higher dimension channel space through a fully connected layer P with a weight matrix of five by twenty real-valued weights.

The paper's improvements: Tom: Now for the results. They trained this model on the generated data for one hundred fifty epochs using a learning rate of zero point zero zero one and a batch size of one. The test loss on their west wind case was really low, at just zero point three percent. That’s actually quite good performance for a complex simulation like this.

Jane: But they also tested how well it generalizes to different conditions. They tested it on winds that were ninety degrees apart, north and south wind directions. The test loss went up to approximately zero point three percent, which is about ten times higher than the west wind case performance.

Lu: That suggests that training only on one direction isn't enough for a model to be robust in the real world where conditions change constantly. But they found something interesting: when they included south wind, which is two hundred seventy degrees, into their training dataset, the test loss dropped substantially from zero point three percent down to zero point three percent again.

Meng: Wait, so adding that one more wind direction improved the performance significantly? That’s a big deal for deployment. It means you don't need massive datasets covering every possible scenario just to get decent accuracy.

Tom: Exactly, and it shows that the model has some inherent ability to learn these different physical relations. Also, they noted that the accumulated error for predicting sequential time steps gets bigger as time goes on, reaching one meter per second within seventy steps.

Jane: That’s a caveat we need to keep in mind: while it’s accurate for instantaneous snapshots, if you try to predict what will happen far into the future, the prediction error grows. That points us toward where we need to focus our next efforts.

Conclusion: Tom: So, summarizing this Fourier neural operator for real-time simulation of three dee dynamic urban microclimate paper: they’ve shown this model can simulate wind fields in three dimensions and in real time with an error rate of less than one percent on the instantaneous spatial velocity magnitude <ref:2308.03985#pg1,Fourier neural operator for real-time simulation of 3D dynamic urban microclimate>.

Jane: The main implication is that we can use AI to speed up the modeling of these complex, non-linear fluid dynamic systems, moving us closer to having quick simulations for urban microclimates.

Lu: They also pointed out a clear path forward: they plan to reduce the temporal prediction error using approaches like attention mechanisms or by incorporating physical constraints so the predictions stick closer to what the governing equations demand.

Meng: From an engineering standpoint, if you want to actually deploy this, you’d need more efficient models that can handle more variables, maybe predicting both the full three-dimensional velocity vector—the u, v, and w components—and also temperature fields.

Lalam: I just think the inclusion of south wind in their training data shows that multi-directional training helps with generalization across different wind patterns. It’s a solid step for real-world applicability.

Wenhui Peng, Shaoxiang Qin, Senwen Yang, Jianchun Wang, Xue Liu, Liangzhu (Leon) Wang

Concordia University · McGill University · Southern University of Science and Technology · The Hong Kong Polytechnic University

cs.LG, cs.CE, cs.NA, math.NA, physics.ao-ph, physics.flu-dyn

Submitted: 2023-08-08

Updated: 2023-09-30

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 83/100

The gist: The gist The Fourier Neural Operator (FNO) model has been applied for real-time three-dimensional (3D) urban wind field simulation, enabling real-time simulation of 3D dynamic urban microclimate

Key concepts

Urban Microclimates: These are localized weather conditions within cities that differ from surrounding areas due to the presence of buildings and urban structures. They significantly impact human comfort, health, and energy efficiency in urban environments.
Computational Fluid Dynamics (CFD): CFD is a numerical method used to simulate fluid flow, such as wind patterns over an urban area. In this study, CFD simulations were used to generate the training data that teaches the AI model how wind behaves in cities.
Fourier Neural Operator (FNO): The FNO is a deep learning technique designed to accelerate solving Partial Differential Equations (PDEs) and modeling fluid dynamics. It learns complex mappings from input data to desired outputs, allowing for very fast predictions of 3D wind fields.

Terminology

Fourier Neural Operator — 2× in this paper
Computational Fluid Dynamics — 1× in this paper
Energy Efficiency — 1× in this paper · explained in 3 episodes

Summary

The gist The Fourier Neural Operator (FNO) model has been applied for real-time three-dimensional (3D) urban wind field simulation, enabling real-time simulation of 3D dynamic urban microclimate <ref:2308.03985#pg2>.

Introduction and Motivation

Global urbanization has underscored the significance of urban microclimates for human comfort, health, and building/urban energy efficiency <ref:2308.03985#pg2>. Understanding local microclimates is essential for cities to prepare for climate change and effectively implement resilience measures <ref:2308.03985#pg2>. Four approaches are commonly used to study urban microclimates: field measurement and meteorological observation, wind tunnel experiments, computational fluid dynamics (CFD) simulation, and recent advancements in artificial intelligence and machine learning techniques <ref:2308.03985#pg2>. The rise of deep learning techniques has opened new opportunities for accelerating the modeling of complex non-linear interactions and system dynamics <ref:2308.03985#pg2>. The Fourier Neural Operator (FNO) has been shown to be very promising in accelerating solving the Partial Differential Equations (PDEs) and modeling fluid dynamic systems <ref:2308.03985#pg2>.

Methodology

The utilization of the FNO model for predicting urban velocity magnitude follows a straightforward workflow where absolute velocity magnitude 3D wind fields for continuous time intervals must first be generated <ref:2308.03985#pg2>. The training and testing data are generated from the computational fluid dynamics (CFD) simulation of the urban area by CityFFD60 <ref:2308.03985#pg2>. The fundamental theory of CityFFD is described in the previous literature62–64 <ref:2308.03985#pg2>. The training data is divided into training samples based on the desired input and output time steps, and the FNO model is trained using this data <ref:2308.03985#pg2>. To accommodate one training sample in the Tesla A100 GPU, downsampling of the original data is necessary through 3rd-order spline interpolation <ref:2308.03985#pg2>. The input tensor a(x) of shape 200 × 200 × 150 × 5 is used, where the first three dimension is the grid mesh in the spatial x, y,z direction and the last dimension 5 is the number of sequential time steps <ref:2308.03985#pg2>.

FNO Model Construction

The Fourier neural operator learns an approximation of G† by constructing a mapping parameterized by θ ∈ Θ <ref:2308.03985#pg2>. The overall architecture of Fourier Neural Operator (FNO) is shown in figure 3 <ref:2308.03985#pg2>. The input a(x) is lifted to a higher dimension channel space by a fully connected layer P, where P is a fully connected layer parameterized by a 5 × 20 real-valued weight matrix <ref:2308.03985#pg2>. Then a sequence of Fourier layers is applied, where the Fourier layer keeps the tensor shape unchanged <ref:2308.03985#pg2>. The core of FNO is the sequence of Fourier layers42 <ref:2308.03985#pg2>. The first Fourier layer takes vi ∈ R 200×200×150×20 as input tensor, and vi is permuted into 20 × 200 × 20 before the Fourier transformation. The weight matrix R is parameterized by a 20 × 8 × 8 × 8 complex-valued tensor.

Results and Discussion

The model was trained on the generated 3D velocity magnitude data for a total of 150 epochs, with a learning rate of 0.001 and a batch size of 1. The test loss on the west wind case demonstrates remarkable accuracy, with a value of 0.3%. To evaluate the model’s generalization capability, it was tested on winds differing by 90 degrees (north and south wind), where the test loss increased to approximately 3%, roughly tenfold compared to the West wind case. The inclusion of south wind (270°) in the training dataset results in a substantial reduction in its test loss, dropping from 3% to 0.3%. The accumulated error for sequential time step prediction grows larger as time progresses, reaching 1 m/s within 70 steps.

Computational Efficiency

The FNO approach provides over 25 times speed up compared to the traditional numerical solver. This speedup allows real-time simulations for 3D dynamic urban microclimate. Table 2 shows that the FNO approach provides over 25 times speed up compared to the traditional numerical solver.

Conclusion and Future Work

Our study demonstrates that the FNO approach has the capacity to significantly expedite the simulation of the 3D urban wind field while maintaining a high level of accuracy in reconstructing the instantaneous spatial velocity magnitude field, with an error rate of less than 1%. The FNO model, trained solely on data of one wind direction, can generalize well to distinct wind directions. In future work, we plan to reduce the temporal prediction error using approaches such as the attention mechanism or incorporating the physical constraints to ensure that the predictions are subject to the governing equations and conservation laws. Information like the size and density of the buildings enables the FNO model to acquire a more comprehensive understanding of the physical relation intrinsic to the environment and thus improve generalization capacity.

Improvements for AI systems

Improve Data Generalization via Multi-Directional Training: The inclusion of south wind (270°) in the training dataset results in a substantial reduction in its test loss, dropping from 3% to 0.3%. This suggests that including data with more wind directions can improve generalization performance, which is vital for real-world urban microclimate prediction where wind direction is highly variable and dynamic.
Enhance Temporal Prediction Robustness via Attention Mechanisms: In future work, we intend to reduce the temporal prediction error using approaches such as the attention mechanism30,55,76,78,79. Implementing attention mechanisms within the FNO architecture will allow the model to weigh important past time steps more effectively when predicting future states.
Increase Predictive Scope via Multi-Variable Output: In future work, we aim to develop more efficient and lightweight models and predict more variables including the velocity components and temperature field. The system can be extended to simultaneously reconstruct the full 3D velocity vector (u, v, w) and the air temperature field alongside wind speed magnitude.