Dictionary learning for Kernel EDMD

arXiv:2604.25572 · math.DS, cs.LG · Submitted 2026-04-28 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Dictionary learning for Kernel EDMD".

Jane: The gist: This paper introduces dictionary learning for kernel extended dynamic mode decomposition (kEDMD) to streamline the process of finding useful kernels for approximating Koopman operators in nonlinear dynamical systems.

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're talking about this paper called "Dictionary learning for Kernel EDMD". Basically, it's trying to make analyzing nonlinear systems easier by turning that nonlinear problem into a linear one using something called the Koopman operator.

Jane: That’s right. But approximating that operator in a finite space with standard EDMD is tricky because you need to pick a fixed list of observables up front. This paper suggests we can define those observables implicitly through kernels instead, which is what they call kEDMD <ref:2604.25572#pg1>.

Lu: What they claim is that dictionary learning for EDMD can be extended to this kernel version, and they use gradient-based optimization over the learnable kernel parameters <ref:2604.25572#pg1>. They show how this approach actually produces useful kernels for the original kEDMD problem.

Meng: From a deployment standpoint, if you have to train a complex kernel, that sounds like it adds complexity to your inference pipeline and the training itself has to be very stable Meng. What's the real mechanism here?

Tom: Well, they simplify things by defining the dictionary as sk theta = g theta(times, x one),, g theta(times, x N), and they show a relationship between the standard EDMD version and this kernel version <ref:2604.25572#pg3>. This connection lets you map between the two versions of kernel EDMD through a proposition <ref:2604.25572#pg1>.

Jane: So, instead of manually picking those functional basis elements, you are learning the function itself, which is defined by a kernel g theta parameterized by theta. The paper focuses on streamlining this by using dictionary learning to optimize those kernel parameters <ref:2604.25572#pg1>.

Lalam: I see the core idea is that they take a weighted list of kernels with random starting values as input, and then output a set of kernels and their parameter values that are suitable for approximating the Koopman operator <ref:2604.25572#pg1>. They also show how you can remove unimportant kernels by analyzing the weights in the weighted sum <ref:2604.25572#pg1>.

Tom: That sounds like a smart way to prune the dictionary, which is crucial because if you have too many observables, you get spectral pollution <ref:2604.25572#pg3>. The experiment on the Duffing oscillator shows that prediction loss and eigenprediction loss agree and decrease rapidly before stabilizing around ten-five <ref:2604.25572#pg4>.

Jane: That stabilization at ten-five is a concrete number, which is really helpful for judging how well the learned representation captures the underlying dynamics <ref:2604.25572#pg4>. It suggests that this method can indeed capture the true eigenvalues of the operator better than starting with random kernels.

Paper summary: Lu: The paper also notes a limitation: they are still working with less smooth kernels, like those from the Matérn family Meng. They suggest future work might involve using eigen-based losses with eigenpairs that have less spectral pollution to improve training <ref:2604.25572#pg1>.

Meng: If we're deploying this, how hard is it to train these kernels on real system data? Is the gradient optimization feasible for large systems?

Tom: They summarize the full algorithm in Algorithm two which involves alternating between computing K and performing a gradient step on theta in Algorithm one <ref:2604.25572#pg1>. The training process uses subsampling of N points from X and Y to create sets and <ref:2604.25572#pg1>.

Jane: So, the main goal is to find a representation—these kernels—that accurately models the system's evolution while being efficient enough for practical use with the Koopman approximation <ref:2604.25572#pg1>. It’s about finding a good balance between accuracy and complexity.

Lalam: My literature review confirms that this approach is specifically designed to handle systems where the number of required observables might be larger than what you have in your snapshot set, which is why they focus on the kernel definition <ref:2604.25572#pg1>. It's a targeted solution for that specific challenge in EDMD.

Tom: So, to wrap up this part, this paper introduces dictionary learning for Kernel EDMD by using gradient-based optimization over kernel parameters to find good representations of the Koopman operator <ref:2604.25572#pg1>. It moves beyond just picking fixed functionals and learns them directly from the data structure itself.

Jane: And it shows that this process can prune unnecessary kernels, leading to more accurate eigenpairs after training on things like the Duffing oscillator <ref:2604.25572#pg4>. This is significant because it suggests a path toward building more efficient, tailored models for complex dynamics.

Lu: The implication for representation learning is that if you can define your dictionary implicitly through a learnable kernel, you bypass the hard choice of what those functionals should be initially <ref:2604.25572#pg1>. It shifts the burden from manual feature engineering to optimization guided by the loss function L dict(theta) <ref:2604.25572#pg1>.

Meng: For production, that means we need to make sure this kernel learning process is fast enough for our typical data throughput, otherwise, it just becomes a slow training bottleneck Meng. We need to see how scalable the gradient steps are when you're dealing with high-dimensional spaces.

Tom: That’s the practical question. The paper shows it works on the Kuramoto-Sivashinsky equation too, which is another nonlinear system <ref:2604.25572#pg4>. It seems robust across different types of dynamics, as long as the system has some underlying structure that a kernel can capture.

Paper summary: Jane: So, for someone listening who doesn't read papers every day, think of it this way: instead of drawing a fixed set of lines to describe how a car moves on the road, you are learning a flexible shape function—a kernel—that describes the car's motion based on its current position and speed <ref:2604.25572#pg1>.

Lalam: That analogy captures the shift perfectly. The paper is about learning that flexible shape function, or kernel, to approximate the system's behavior in a way that respects the structure of the data itself <ref:2604.25572#pg1>.

Tom: So, we've covered what it is: dictionary learning for Kernel EDMD and how it uses gradient optimization to find those kernels. It’s a method that learns useful functional representations for the Koopman operator by treating the kernel parameters as trainable variables <ref:2604.25572#pg1>.

Jane: And the main result is that this method can prune unimportant components, leading to more accurate results when applied to systems like the Duffing oscillator <ref:2604.25572#pg4>. It’s a step toward more efficient and principled data-driven modeling of nonlinear dynamics.

Lu: The real change this suggests is that we can move away from manually designing dictionaries for EDMD and instead let the optimization process discover the best way to represent the state space evolution <ref:2604.25572#pg1>. It's an interesting direction for representation learning in physics too.

Meng: I still see it as a research tool right now, because training these complex kernel models on massive, noisy datasets is going to be a huge computational cost to manage at scale Meng. Until we see a deployment where this is significantly faster than standard EDMD, it’s more of an interesting proof-of-concept for representation quality.

Tom: For the next step in research, they are looking at how to improve the training itself, specifically by using eigen-based losses with eigenpairs that have less spectral pollution <ref:2604.25572#pg1>. That’s where we might see the next major refinement.

Jane: So, while it’s a strong method for finding good kernels, their next challenge is dealing with the inherent noise or artifacts in those kernels when training on certain system types like smoother ones <ref:2604.25572#pg1>. It keeps pushing the boundaries of what we can learn from dynamical data.

Lalam: The paper's focus is clearly on improving the quality of the kernel approximation itself, specifically by using those eigenpairs to guide the loss function better <ref:2604.25572#pg1>. It’s an iterative improvement cycle focused on refining the learned functional basis <ref:2604.25572#pg1>.

Conclusion: Tom: So, we've been looking at this paper called "Dictionary learning for Kernel EDMD." The core idea is using dictionary learning to figure out the best kernels for approximating Koopman operators in nonlinear systems.

Jane: Yeah, and what that means is they aren't just picking a fixed set of functions; they are learning the actual shape of those functions from the data.

Tom: Exactly. They take a weighted list of random kernels as input and optimize them using gradient descent to get useful representations for those dynamics. The authors show you can even prune unimportant kernels by looking at their weights in that weighted sum.

Jane: That pruning part is interesting because it suggests you don't need a huge dictionary to capture the right physics, which makes the whole process more efficient.

Tom: It's about finding a good balance between accuracy and complexity when you’re building that infinite-dimensional Koopman approximation in finite dimensions.

Jane: And what this really means for someone listening is that instead of spending all their time picking those initial functional bases, they can let the data guide what the system actually looks like.

Tom: It shifts the burden from manual feature engineering onto an optimization process that finds representations tailored to the specific nonlinear system, like the Duffing oscillator.

Jane: Right, it's a step toward building more principled ways to model complex dynamics without getting stuck with an arbitrary set of starting points.

Tom: Next up, we’re going to look at what they found in their experiments on those real systems and how those results translate into practical use for folks actually deploying these models.

Erik Lien Bolager, Boumediene Hamzi, Houman Owhadi, Ioannis G. Kevrekidis, Felix Dietrich

School of Computation, Information and Technology, Munich Data Science Institute, Munich Center for Machine Learning, Technical University of Munich, Germany · Department of Computing and Mathematical Sciences at Caltech, USA · The Alan Turing Institute at the Alan Turing Institute in the United Kingdom

math.DS, cs.LG

Submitted: 2026-04-28

Updated: 2026-10-14

Journal ref: Chaos:An.Interdisc.Jou.of.Nonlin.Sci. 36 (2026) 103123-1-20

DOI: 10.1063/5.0341144

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 76/100

The gist: The gist: This paper introduces dictionary learning for kernel extended dynamic mode decomposition (kEDMD) to streamline the process of finding useful kernels for approximating Koopman operators in

Key concepts

Koopman Operator: This is a mathematical tool used to linearize complex nonlinear dynamical systems. It allows researchers to analyze nonlinear behavior by transforming it into a linear problem in an infinite-dimensional space, making it easier to find patterns and predictions.
Extended Dynamic Mode Decomposition (EDMD): A common algorithm used to approximate the Koopman operator in finite dimensions. It works by finding a dictionary of basis functions that can describe the system's state evolution, allowing the nonlinear dynamics to be approximated by a linear relationship between these basis functions.
Kernel Extended Dynamic Mode Decomposition (kEDMD): A variation of EDMD where the dictionary is implicitly defined using a kernel function. This approach is useful when a large number of observables are needed but only a limited set of system snapshots are available, helping to define the necessary basis functions without explicitly choosing them.

Terminology

Loss Function — 45× in this paper · explained in 2 episodes
Koopman Operator — 38× in this paper · explained in 4 episodes
Regularization — 26× in this paper · explained in 3 episodes
Pruning — 20× in this paper · explained in 4 episodes
Hyperparameters — 7× in this paper · explained in 2 episodes
Kernel Extended Dynamic Mode Decomposition — 5× in this paper
Extended Dynamic Mode Decomposition (EDMD) — 4× in this paper · explained in 3 episodes
Fine-Tuning — 4× in this paper · explained in 5 episodes
Hilbert Space — 4× in this paper · explained in 2 episodes
Elias — 3× in this paper · explained in 2 episodes
Generalization — 3× in this paper · explained in 13 episodes
Accuracy — 2× in this paper · explained in 1 episode
Core — 2× in this paper · explained in 2 episodes
Feature Engineering — 2× in this paper · explained in 3 episodes
Generator — 2× in this paper · explained in 2 episodes
Representation Learning — 2× in this paper · explained in 3 episodes
Activation Function — 1× in this paper · explained in 1 episode
American Control Conference — 1× in this paper
Dynamic Mode Decomposition (DMD) — 1× in this paper · explained in 4 episodes
Embeddings — 1× in this paper · explained in 2 episodes
Feature Extraction — 1× in this paper · explained in 3 episodes
Interpretability — 1× in this paper · explained in 15 episodes
Papers — 1× in this paper · explained in 3 episodes
Sparsity — 1× in this paper · explained in 4 episodes
Stochastic Gradient Descent (SGD) — 1× in this paper · explained in 5 episodes
Uncertainty Propagation — 1× in this paper · explained in 3 episodes

Summary

The gist: This paper introduces dictionary learning for kernel extended dynamic mode decomposition (kEDMD) to streamline the process of finding useful kernels for approximating Koopman operators in nonlinear dynamical systems.

Introduction and Motivation

Kernel Extended Dynamic Mode Decomposition (kEDMD)

The problem arises because implicitly defining the dictionary via a kernel may be a solution when the system requires many more observables than snapshots. In kEDMD, we start by applying SVD to Ψ(X) = Q˜Σ˜Z˜T and then reducing it to ΨP = QΣZ T. This reduction allows one to define the Koopman approximation similarly to Equation (1), i.e., KP = arg min K̃ ∈RM×M∥PΨ(Y) − (P K˜)PΨ(X)∥2F. The key part of the equation above is that Kˆ ∈ R N×N, and if one only needs to compute Kˆ, the high dimensionality of M is avoided. To compute Kˆ, we need to compute Z, Σ, and ((PΨ(X))T(PΨ(Y))), which are matrices induced by inner products between the dictionary elements.

Dictionary Learning for kEDMD

To perform dictionary learning for kEDMD, we assume a kernel gθ is parameterized by θ and define a loss function Ldict(θ) = ∥PΨθ(Y) − KP (PΨθ(X))∥2F. The paper shows how to evaluate this loss function using properties of KP and Kˆ. This leads to the loss function Ldict(θ) = ∥PΨθ(Y) − KP PΨθ(X)∥2F = ∥VˆWˆ Σ +Z T gθ(Y, X) − VˆΛWˆ ΣZ T∥2F.

Simplifying kEDMD and Loss Functions

To make dictionary learning for kEDMD applicable, we simplify the method by defining a dictionary as Ψskθ = gθ(·, x1),..., gθ(·, xN). Applying standard EDMD now gives us K = Ψskθ(Y)Ψskθ(X) + = gθ(Y, X)gθ(X, X) +, (10) <ref:2604.255723,This relationship shows that Ktr = (ZΣ)+KskZΣ <ref:2604.25572#pg4>. This connection allows us to map between the two versions of kernel EDMD via Proposition 2.

Experimental Results

Experiments on the Duffing oscillator show that the prediction loss and eigen prediction loss agree and decrease rapidly before stabilizing around 10−5 <ref:2604.255723,In Figure 1, we calculate the mean loss over all five batches for each epoch <ref:2604.25572#pg4>. The parameter found by kEDMD-DL captures the underlying dynamics well, even when training starts with a kernel that clearly does not. After pruning, the number of eigenvalues with high residuals is lower, and therefore the eigenpairs produced after training are a more accurate representation of the operator’s true eigenvalues.

Conclusion and Future Work

The method approximates the system dynamics while pruning unimportant kernels to find accurate and efficient kernel representations. Current limitations include training on less smooth kernels, such as those from the Matérn family. Future work suggests investigating resDMD into the training itself by computing eigen-based losses using eigenpairs with less spectral pollution.

Algorithm Summary

The full algorithm is summarized in Algorithm 2, using only Lsk pred as the loss function. This procedure involves alternating between computing K and performing a gradient step on θ in Algorithm 1. The training involves subsampling N˜ points from X and Y to create the sets X˜ ⊆ X and Y˜ ⊆ Y. The final loss functions are defined by dividing by the integral of the dictionary.

Kernel Selection Strategy

The paper proposes a weighted sum of kernels gθ = X Σi=1 w˜2i exp −∥x − y∥2σ2i + ˜w24 cos(ax − y2) + ˜w5(c 21⟨x, y⟩)

Improvements for AI systems

textbfProcessing via Kernel Extended Dynamic Mode Decomposition (kEDMD-DL): Enabling Implicit Dictionary Learning for Koopman Operators. The improved system can implicitly define a dictionary through a kernel and then perform gradient-based optimization over learnable kernel parameters, leading to useful kernels for the original kEDMD. This allows the system to approximate the Koopman operator using a weighted sum of kernels, where unimportant ones can be removed by analyzing the weights in the weighted sum.
textbfHandling High-Dimensional Function Spaces: Utilizing Kernel Embeddings for System Analysis. The system can map states into a high-dimensional space where it treat[s] the system in a linear fashion, leveraging kernels to approximate this manifold, which is crucial for studying complex nonlinear dynamical systems beyond finite-dimensional phase spaces.
textbf Robust Model Selection via Multi-Kernel Optimization. The AI can select the optimal set of kernel components by optimizing outer parameters using a loss function that includes a weighted sum of kernels, allowing it to identify the useful kernels in a sum and prune unimportant ones based on their contribution to the dynamics.
textbf Adaptive Training Strategies via Loss Function Selection. The system can dynamically choose between different learning objectives, such as switching between dictionary loss, eigen prediction loss, and eigen prediction loss (Lsk pred), depending on the stage of training or the specific problem (e.g., trajectory prediction versus spectral analysis).
textbf Efficient Computation via Subsampling and Pruning. To manage computational bottlenecks when dealing with large datasets or complex PDEs like the Kuramoto-Sivashinsky equation, the system can employ subsampling of snapshots (subsample N˜ points from X and Y) and iteratively prune kernels based on their weights, as demonstrated by the strategy: training first on a smaller N˜ and then fine-tuning on the larger N˜.

Abstract

Studying nonlinear dynamical systems through their state space behavior can be challenging, and one possible alternative is to analyze them via their associated Koopman operator. This turns the nonlinear problem into a linear, infinite-dimensional one. To approximate the operator in finite dimensions, extended dynamic mode decomposition (EDMD) is a commonly used algorithm. It requires a finite list of functionals and a set of snapshots from the system to compute an approximation of the operator and its corresponding spectrum. Instead of choosing the list of functionals directly, it can be implicitly defined via kernels, a method known as kernel extended dynamic mode decomposition (kEDMD). However, one still needs to define the kernel and choose its parameter values. In this paper, we aim to streamline this process by extending dictionary learning for EDMD to kernel learning in kEDMD. By simplifying kEDMD, we show how to perform gradient-based optimization over the learnable kernel parameters and demonstrate that this method leads to useful kernels for the original kEDMD. The focus of our work is a method that takes a weighted list of kernels with randomly initialized values as input and outputs a list of kernels and parameter values suitable for approximating the Koopman operator of the underlying system. We demonstrate that unimportant kernels can be removed from the list by analyzing the weights in the weighted sum. We evaluate the method across several experiments, including the Duffing oscillator and the Kuramoto-Sivashinsky equation, showcasing the method's different strengths.