Building causation links in stochastic nonlinear systems from data

arXiv:2509.07701 · cond-mat.stat-mech, cs.LG · Submitted 2025-09-09 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Building causation links in stochastic nonlinear systems from data".

Jane: Here's the gist of the paper. The gist: This work addresses how to detect intrinsic causal links in complex stochastic nonlinear systems from data by leveraging machine learning within the framework of response theory.

Tom: Causal relationships are fundamental for making informed decisions and predicting outcomes, but deciphering them from observational data is difficult because correlations alone do not imply causation <ref:2509.07701#pg2>.

Jane: The paper seeks to establish causal links in a large class of complex systems by combining theoretical ideas from response theory with state-of-the-art machine learning techniques to build models from data <ref:2509.07701#pg2>.

Tom: While statistical analysis of causality assumes immediate effects, physics suggests that intervention on a system may cause a response only after some time, making the use of response theory particularly interesting for problems where laboratory experiments are not possible <ref:2509.07701#pg2>.

Title and authors: Tom: Let's talk about the authors for a second. We’ve got Sergio Chibbaro from Université Paris-Saclay, Cyril Furtlehner from INRIA-Saclay, Théo Marchetta also from INRIA-Saclay, Andrei Tiberiu Pantea also at Université Paris-Saclay, Davide Rossetti from Politecnico di Torino and more folks.

Jane: It’s a really interesting mix of physics and computer science folks working together on this problem. They bring a strong background in dynamical systems theory to tackle this data-driven causality challenge.

Lu: Their combined expertise seems aimed at bridging that gap between the theoretical concepts of response theory and the practical application of machine learning techniques for inferring these links from time series data.

Meng: From an engineering standpoint, having researchers from different fields working on this suggests they’re aiming for a robust method that can handle a lot of messy, real-world data, not just clean textbook examples.

Lalam: I see them focusing on how to make interventional causality—that cause-and-effect where you actively perturb something—more accessible when we only have observational time series.

Tom: Exactly. They are pushing the idea that we need a way to define causality that aligns better with how the physical world actually works, rather than just statistical measures of predictive power.

Jane: So, what’s the core concept they are trying to establish? It seems like they are trying to formalize what it means for one variable to be influenced by another during dynamics without needing those underlying evolution laws.

The paper's summary: Tom: The summary of this paper is about establishing a framework that uses response theory, which is connected to the general definition of causality when the system admits an invariant measure and the perturbation is small enough. They define the response of a dynamical system by measuring the difference between trajectories in original dynamics and in a perturbed one, averaged over many realizations.

Jane: That sounds like they are trying to use that idea of response—how things change when you introduce a small nudge—as a way to get closer to the stronger physics-oriented definition of interventional causality, which states that an external action on one variable results in a change in another at some later time.

Lu: They formalize this with the linear response matrix R(t), which is given by the difference between trajectories divided by the perturbation, and for a stationary process, it simplifies to something involving the log of the probability density function rho of x.

Meng: When they move to data-driven modeling, they apply machine learning techniques to estimate this response function in various regimes—both linear and non-linear systems—with or without cofounders present in the data.

Lalam: For linear systems, they suggest two main ways to estimate that response: either estimating the Markov matrix A from data through Markov regression or directly computing it by estimating the auto-correlation of the process.

Tom: And they mention using L1 regularization, like Lasso, to help filter out those small spurious regression coefficients when they are doing that estimation.

Jane: For non-linear generalizations, they look at methods like Non-Linear Multiple Regression, where you use a dictionary of candidate functions and L1 regularization enforces sparsity to keep the model simple.

The paper's improvements: Tom: One major improvement they suggest is using Reservoir Computing, which is a purely data-driven approach that projects a non-linear system into a higher dimensional space so its dynamics look approximately linear, allowing linear regression techniques to be applied.

Lu: That’s interesting because it suggests that even for non-linear systems, if you can map them to a simpler space, you might be able to apply more standard statistical tools to estimate the response function accurately.

Jane: The paper also investigates what happens when we deal with chaotic systems, like the Lorenz 'sixty-three model, where fully nonlinear dynamics occur and the linear approximation of the response function simply fails.

Meng: In those chaotic cases, they suggest using things like a Nested Neural Network or Reservoir Computing to reconstruct phase dynamics in detail, though they admit this only gives a qualitative account of the full response.

Lalam: And for chaotic systems specifically, they introduce Sparse Identification of Nonlinear Dynamical systems, or SINDy, which tries to approximate the system as a linear combination of simpler nonlinear functions inspired by governing equations.

Tom: That SINDy idea sounds promising because it lets you infer the actual model structure from the data itself rather than having someone tell you what equations to look for beforehand.

Jane: The paper also gets into some big theoretical stuff, looking at how the efficiency of a linear response predictor is characterized in the asymptotic limit when dealing with an extensive number of time series using Random Matrix Theory.

Conclusion: Tom: So, to wrap up this discussion on "Building causation links in stochastic nonlinear systems from data," the paper shows that while linear systems let us extract response very efficiently from data, non-linear cases really need physics-guided machine learning approaches for near-exact accuracy.

Jane: And for chaotic systems where the linear approximation breaks down, they suggest reconstruction models like SINDy can infer the correct model structure directly from the data using sparse identification.

Lu: The paper stresses that the response of a system is a delicate statistical observable, so there’s definitely special care required when analyzing responses in complex systems.

Meng: I’m thinking about deployment here; getting that kind of accuracy from a model might be tough if you run it on massive traffic because the methods they propose seem quite computationally intensive for production use right now.

Lalam: I’m fairly confident in their findings because they are grounding the statistical measures in response theory, giving them a solid physical foundation instead of just being purely data-driven guesses.

Tom: So, we’ve seen how they try to move from observational causality to a more interventional view using response theory and ML tools like SINDy for non-linear dynamics.

Jane: It’s a deep dive into making sure the AI we build isn't just finding correlations, but actually understanding the dynamic influence one variable has on another.

Lu: This work gives us a new lens through which to approach complex systems where we don't have perfect mathematical laws describing everything.

Meng: We’ll be watching how these estimation methods scale up when we move from these smaller models to systems with millions of data points or nodes in a network.

Sergio Chibbaro, Cyril Furtlehner, Théo Marchetta, Andrei-Tiberiu Pantea, Davide Rossetti

Université Paris-Saclay · CNRS · INRIA-Saclay

cond-mat.stat-mech, cs.LG

Submitted: 2025-09-09

Updated: 2025-09-09

Comments: 24 pages, 11 Figures. Comments are welcome

Journal ref: Phys. Rev. E 113, 044114, 2026

DOI: 10.1103/kdyd-tc6n

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 76/100

The gist: The gist: This work addresses how to detect intrinsic causal links in complex stochastic nonlinear systems from data by leveraging machine learning within the framework of response theory.

Key concepts

Interventional Causality
This approach defines causality by testing what happens when you deliberately change a variable at time zero and measuring the resulting change later. It is favored because it directly models physical cause-and-effect, distinguishing it from simple observational correlations.
Response Theory
This framework connects causality to how a system reacts when slightly perturbed. It calculates the response function by comparing trajectories of the original system against a slightly altered version, averaged over many scenarios to find the underlying dynamics.
Sparse Identification of Nonlinear Dynamical Systems (SINDy)
This method is used for non-linear systems where standard linear approximations fail. SINDy attempts to reconstruct the system's governing equations by finding a sparse linear combination of simple nonlinear functions, making complex dynamics more understandable from data.
Random Matrix Theory (RMT)
RMT is used in the large-scale limit of time series to measure how efficient causal predictors are. It provides analytical formulas to determine the asymptotic bias and variance of data-driven causal estimators when dealing with extensive amounts of data.

Terminology

Summary

The gist: This work addresses how to detect intrinsic causal links in complex stochastic nonlinear systems from data by leveraging machine learning within the framework of response theory.

Introduction and Motivation

Causal relationships are fundamental for making informed decisions and predicting outcomes, but deciphering them from observational data is difficult because correlations alone do not imply causation <ref:2509.07701#pg2>. The paper seeks to establish causal links in a large class of complex systems by combining theoretical ideas from response theory with state-of-the-art machine learning techniques to build models from data <ref:2509.07701#pg2>. While statistical analysis of causality assumes immediate effects, physics suggests that intervention on a system may cause a response only after some time, making the use of response theory particularly interesting for problems where laboratory experiments are not possible <ref:2509.07701#pg2>. The problem is formulated as determining whether the behavior of one variable has been influenced by another during dynamics without knowing the underlying evolution laws <ref:2509.07701#pg2>.

Definitions of Causality Approaches

The paper distinguishes between two broad types of approaches to causal inference: observational and interventional causality <ref:2509.07701#pg2>. For the observational setting, methods like Granger causality (GC) and transfer entropy (TE) are used to extract information, but these methods rely on selecting the right variables, which is not satisfactory from a physical point of view <ref:2509.07701#pg2>. Interventional causality is favored because it is more easily linked to the physical cause-effect relationship, stating that a perturbation of a variable at time 0 causes a change in another variable at time t > 0 <ref:2509.07701#pg2>. This interventional causality can be mathematically formalized by the relation where there is a causal relation if, given a smooth function F(x), the relationship holds: δF(xkt) / δxj0 ≠ 0 <ref:2509.07701#pg2>.

Response Theory Framework

The response theory connects to the general definition of causality when the system admits an invariant measure and the perturbation is small enough <ref:2509.07701#pg2>. The response of a dynamical system is computed by measuring the difference between trajectories in the original dynamics and in a perturbed one, averaged over many realizations <ref:2509.07701#pg2>. For a stationary process characterized by an invariant probability density function ρ(x), the linear response matrix R(t) is given by Rj→k(t) ≡ lim δxj0→0 δxkt / δxj0 = - (xkt / x0) ∂ ln ρ(x) / ∂xj <ref:2509.07701#pg2>. For a linear Markov system, this relation simplifies to Ri→j (t) = [A t]ji = [CtC −1 0]ji, where Ct is the covariance matrix <ref:2509.07701#pg2>.

Data-Driven Modeling and Response Estimation

The work applies machine learning to estimate the response function in various regimes, including linear and non-linear with or without cofounders <ref:2509.07701#pg2>. For linear systems, the response can be estimated by either estimating the Markov matrix A from data (Markov regression) or directly computing it by estimating the auto-correlation of the process <ref:2509.07701#pg2>. This estimation can be regularized using Lasso regularization (L1 penalty) to filter out small spurious regression coefficients <ref:2509.07701#pg2>. For non-linear generalizations, methods include Non-Linear Multiple Regression, where a dictionary of candidate functions is used and L1 regularization enforces sparsity <ref:2509.07701#pg3>. Reservoir Computing (RC) is another purely data-driven approach that projects a non-linear system into a higher dimensional space to make its dynamics approximately linear, which allows for the application of linear regression techniques <ref:2509.07701#pg3>.

Analysis in Nonlinear and Chaotic Systems

When dealing with non-linear systems, the paper investigates approaches such as using a Nested Neural Network or Reservoir Computing to estimate the response function <ref:2509.07701#pg3>. For chaotic systems like the Lorenz '63 model, where fully nonlinear dynamics occur, the linear approximation fails entirely and offers no insight into actual causality <ref:2509.07701#pg2>. In this case, a properly parameterized neural network can successfully reconstruct phase dynamics in detail but only delivers a qualitative account of the full response <ref:2509.07701#pg3>. Sparse Identification of Nonlinear Dynamical systems (SINDy) is shown to perform outstandingly when approximating the function as a linear combination of simpler nonlinear functions inspired by governing equations <ref:2509.07701#pg3>.

Asymptotic Efficiency and Random Matrix Theory

In the asymptotic limit corresponding to an extensive number of time series, the efficiency of the LRR causal predictor is characterized using Random Matrix Theory (RMT) <ref:2509.07701#pg3>. The empirical L2 regularized linear response reads R(N)(t) = αC(N)t G(N), where G(N) is the resolvent <ref:2509.07701#pg3>. The asymptotic estimation of the causal response is given by (C3), which involves deterministic terms corresponding to the bias and variance, determined analytically through standard random matrix formulas <ref:2509.07701#pg3>. This allows for the determination of T P(h, t) and F P(h, t) for a causal link given an arbitrary threshold h <ref:2509.07701#pg3>.

Conclusion

The study demonstrates that while linear systems allow for efficient extraction of response from data, nonlinear cases require physics-guided machine learning approaches to achieve near-exact accuracy <ref:2509.07701#pg3>. For chaotic systems, the linear approximation fails, but reconstruction models like SINDy can infer the correct model directly from data using sparse identification <ref:2509.07701#pg3>. The response of a system is a particularly delicate statistical observable that requires special care when analyzing responses in complex systems <ref:2509.07701#pg3>.

Acknowledgements

The project has received funding by French ANR grant SPEED (ANR-20-CE23-0025-01) and the ANR grant SCALP (ANR-24-CE23-1320) <ref:2509.07701#pg2>. The paper also cites numerous foundational works in statistics, physics, and machine learning <ref:2509.07701#pg2>.

References

[1] Erik Aurell and Gino Del Ferraro. Causal analysis, correlation-response, and dynamic cavity. In Journal of Physics: Conference Series, volume 699, page 012002. IOP Publishing, 2016 <ref:2509.07701#pg2>.

[2] Aristotle. The complete works of Aristotle. Princeton University Press Princeton, 1984 <ref:2509.07701#pg2>.

[3] David Hume. A treatise of human nature. Clarendon Press, 1896 <ref:2509.07701#pg3>.

[4] Bertrand Russell. On the notion of cause. In Proceedings of the Aristotelian society, volume 13, pages 1–26. JSTOR, 1912 <ref:2509.07701#pg3>.

[5] Katerina Hlav´aˇckov´a-Schindler, Milan Paluˇs, Martin Vejmelka, and Joydeep Bhattacharya. Causality detection based on information-theoretic approaches in time series analysis. Physics Reports, 441(1):1–46, 2007 <ref:2509.07701#pg2>.

[6] Judea Pearl. Causality. Cambridge university press, 2009 <ref:2509.07701#pg3>.

[7] David D Zhang, Harry F Lee, Cong Wang, Baosheng Li, Qing Pei, Jane Zhang, and Yulun An. The causality analysis of climate change and large-scale human crisis. Proceedings of the National Academy of Sciences, 108(42):17296–17301, 2011 <ref:2509.07701#pg3>.

[8] Umberto Marini Bettolo Marconi, Andrea Puglisi, Lamberto Rondoni, and Angelo Vulpiani. Fluctuation–dissipation: response theory in statistical physics. Physics reports, 461(4-6):111–195, 2008 <ref:2509.07701#pg3>.

[9] Hong-Li Zeng, Erik Aurell, Mikko Alava, and Hamed Mahmoudi. Network inference using asynchronously updated kinetic ising model. Physical Review E, 83(4):041135, 2011 <ref:2509.07701#pg3>.

[10] Rudolf Friedrich, Joachim Peinke, Muhammad Sahimi, and M Reza Rahimi Tabar. Approaching complexity by stochastic methods: From biological systems to turbulence. Physics Reports, 506(5):87–162, 2011 <ref:2509.07701#pg3>.

[11] Marco Baldovin, Fabio Cecconi, Massimo Cencini, Andrea Puglisi, and Angelo Vulpiani. The role of data in model building and prediction: a survey through examples. Entropy, 20(10):807, 2018 <ref:2509.07701#pg3>.

[12] Federica Ferretti, Victor Chardes, Thierry Mora, Aleksandra M Walczak, and Irene Giardina. Building general Langevin models from discrete datasets. Physical Review X, 10(3):031018, 2020 <ref:2509.07701#pg3>.

[13] Jonathan S Yedidia, William T Freeman, Yair Weiss, et al. Understanding belief propagation and its generalizations.

Improvements for AI systems

  1. Improved causal inference for large scale Markov process networks by leveraging state of art ML techniques to build models from data, enabling detection of intrinsic causal links in complex systems. This allows AI systems to identify how changes in one variable influence others without needing underlying evolution laws or laboratory experiments.

  2. Development of a physics-guided approach for inferring causal links between dynamical variables using the linear response of the system as a proxy for interventional causality experiments. This enables AI to perform causal analysis on observational data, specifically by computing the response function Rj→i(t) based on perturbation methods like Eq. (9).

  3. Creation of a sparse regularized neural network approach for reconstructing Markov processes and response functions, which is particularly efficient with a very small number of parameters needed. This allows AI to build accurate models of complex dynamics even with limited training data by filtering out spurious regression coefficients using L1 regularization.

  4. Implementation of a hybrid dual-channel architecture combining linear transformations and nonlinear mappings via nested sub-networks for predicting the next state of variables (x, y, z). This system can capture latent interactions and nonlinear dependencies that would be inaccessible to purely linear models while maintaining interpretability through separate channels for linear and nonlinear signal flow.

  5. Application of the proportional scaling limit analysis based on Random Matrix Theory (RMT) to characterize the asymptotic efficiency of causal predictors in large scale causal graphs with extensive numbers of time series. This allows AI systems to quantify the performance, including bias and variance, of data-driven causal predictors like LRR under conditions where both number of nodes and time steps go to infinity.

Sources

Related papers