Generalized Matheron Variational Implicit Processes

arXiv:2610.07938 · cs.LG · Submitted 2026-10-06 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: Today's paper: "Generalized Matheron Variational Implicit Processes".

Jane: Detailed Research Summary: Generalized Matheron Variational Implicit Processes (GMVIP) The provided text details Generalized Matheron Variational Implicit Processes (GMVIP),

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So we're looking at this paper, "Generalized Matheron Variational Implicit Processes," and it's trying to tackle something specific about modeling functions when you don't know the function space density directly.

Jane: That sounds like a tough spot, Tom. What does that actually mean for someone who is building these models?

Tom: Well, the core idea here is using this method to get posterior samples even when we can’t write down the probability of a specific function. It introduces something called Generalized Matheron Variational Implicit Processes as a pathwise variational family for those priors.

Lu: What I find interesting is how they construct these posterior samples by drawing a function from the prior and then applying this correction anchored at some inducing inputs.

Meng: So, if you don't know the function space density, how does that "correction" help us get useful samples? Does it just make the model more accurate?

Jane: It seems to be about making sure the posterior samples retain the actual structure and variability of that original implicit process, which is a big deal when things aren't simple Gaussian distributions.

Tom: Exactly. They define this construction with a scalar output formula, f(times) = g(times) + Z(times)

mu + D a - g(Z): , where g is a sample from the prior function distribution.

Lu: And they use an empirical operator, Z, which they estimate from an empirical prior-sample covariance, making it similar to what you see in Gaussian Process Matheron conditioning.

Meng: If that operator is based on the samples we already have, does that mean we can get away from needing a full density estimation of the function space?

Jane: That’s the point. They state that this empirical covariance construction preserves the prior mean and covariance in the population limit when using general implicit priors.

Tom: That's a strong claim because it means even if your prior isn't Gaussian, you still keep those first two moments of the original implicit process intact.

Lu: And they show that for Gaussian priors, this construction exactly recovers the standard inducing-variable variational GP construction.

Jane: So they have this nice property where it works perfectly for standard setups, but it keeps working even when things get more complex with non-Gaussian processes.

Tom: Right. They also provide a result showing that for any coefficient law, q phi(a), the induced posterior is defined as q(f) = Z p zero(fu) q phi(u) du <ref:2610.07938#pg1>.

Title and authors: Meng: That sounds like a way to simplify the posterior structure by focusing on the inducing inputs rather than trying to map out every single function.

Jane: It suggests that you can use these inducing points to capture the essential dynamics of the function space, which is crucial for scalability.

Tom: The training loop they describe involves drawing a path a(r) about q phi(a) and a coherent prior sample g(r) about p zero(g) at the combined set of inducing locations X B Z <ref:2610.07938#pg1>.

Lu: Then you compute an augmented term, u(r) from mu + D a(r), which is then used in the posterior sample construction: f(r) B from g(r)(X B) + Z(X B)u(r-g)(Z).

Jane: The loss function they use is based on a term that compares the posterior sample to a target, which involves the KL divergence between q phi(a) and some distribution related to the inducing coefficients.

Tom: And the optimization uses Adam step on that loss function, -L bt, to update those parameters phi and psi.

Meng: From a deployment standpoint, how much computational cost are we looking at here? This sounds like it might be heavy if you have many prior samples.

Jane: The paper does give a cost estimate for constructing the empirical operator Z, which is proportional to O C bank(N + M, B) + BNM + BM squared + NM squared + M cubed <ref:2610.07938#pg1>.

Tom: After that setup, one posterior sample costs about O C g(N + M) + M squared + NM <ref:2610.07938#pg1>. It's definitely more than a simple coefficient-only VIP method.

Lu: But the paper notes that this overhead is justified because it allows for constructing a Matheron operator from prior-bank evaluations, which is a key mechanism.

Jane: And they show that for Gaussian priors, it recovers standard inducing-variable GP constructions, which gives us a solid baseline to compare against.

Tom: But the real meat seems to be in Proposition four which shows the empirical Matheron operator preserves the first two moments of the original implicit process even when the prior is not Gaussian <ref:2610.07938#pg1,of the original implicit process>.

Meng: So if we use this setup on some messy data, we still maintain some fundamental statistical properties of what we are modeling.

Jane: That’s a strong implication because it means you don't lose that population structure just because you moved away from simple Gaussian assumptions in the prior.

Tom: They also mention that for any coefficient law q phi(a), the induced posterior is defined as q(f) = Z p zero(fu) q phi(u) du, which simplifies things a bit <ref:2610.07938#pg1>.

Title and authors: Lu: They also point out that they can recover standard inducing-variable GP constructions exactly for Gaussian priors, and for any coefficient law q phi(a), it shows the induced posterior is defined in that specific way.

Jane: So this paper offers a way to bridge the gap between simple, well-understood models and the more complex implicit processes we actually encounter in AI.

Tom: And they also show that for Gaussian priors, it recovers standard inducing-variable GP constructions, and for any coefficient law q phi(a), it shows that the induced posterior is q(f) = Z p zero(fu) q phi(u) du <ref:2610.07938#pg1>.

Meng: It sounds like a lot of machinery to build upfront, but the performance metrics on various tasks are quite competitive.

Jane: And they actually show strong empirical performance in regression and classification, achieving the lowest mean RMSE and CRPS in some comparisons.

Tom: Prior learning is crucial here; tunable-prior GMVIP consistently outperforms its frozen counterpart on every dataset and metric reported, which tells us the prior matters a lot.

Lu: They also highlight that increasing the inducing basis size S generally improves point forecasting accuracy, but they don't guarantee monotone gains across all metrics.

Jane: And there’s this idea about the residual path acting as both an additional source of predictive uncertainty and a structural regularizer against overfitting the finite-dimensional inducing representation.

Tom: That residual path is interesting because removing it leads to increased KL divergence in the inducing coefficient space, suggesting it carries significant information.

Meng: So, if we want to combat overfitting on things like Boston or Yacht datasets, keeping that residual path seems like a practical way to go.

Jane: It's definitely a feature they highlight as something that helps mitigate severe overfitting when working with these implicit process priors.

Tom: But we also have limitations. The paper says the empirical operator Z is sensitive to the quality of the prior bank, and poor choices can degrade performance.

Lu: And they also note that for non-Gaussian or highly structured processes, this method performs inference under a surrogate prior induced by the replacement map that might differ significantly from the original implicit-process prior.

Jane: So while it's flexible, we have to be careful because of how much it relies on those initial prior samples you feed into the system.

Title and authors: Tom: And there’s the computational cost, which is proportional to O C bank(N + M, B) + BNM + BM squared + NM squared + M cubed <ref:2610.07938#pg1>. It's not something you can deploy with minimal hardware.

Meng: So if we want to use this at scale on real traffic data, we need to be very mindful of that setup cost and the dependency on those prior evaluations.

Jane: And they also mention that they compare their empirical GMVIP method against coefficient-only VIP-style methods, which are computationally cheaper but lack the Matheron operator construction capability.

Tom: So the trade-off is clear: you get this more complex machinery for better representation learning, but it comes with a higher computational price tag compared to simpler coefficient methods.

Lu: The ablation studies also show that freezing the prior removes train–test discrepancy but typically leads to underfitting both splits.

Meng: So tuning the prior seems like a necessary evil here; you have to balance fitting the training data against maintaining generalization on unseen data.

Jane: And they suggest that increasing the number of prior samples, for instance, B = two thousand forty-eight can improve metrics like SW2 and W1 sharply, which suggests that a larger bank is beneficial for aggregate results <ref:2610.07938#pg1>.

Tom: So to wrap up the main points on Generalized Matheron Variational Implicit Processes, it's a sophisticated framework that uses pathwise constructions with an empirical Matheron correction.

Lu: It successfully addresses modeling implicit-process priors by introducing this correction anchored at inducing locations and shows how it preserves the structure and variability of the original process.

Jane: Overall, this paper provides a solid framework for posterior inference when you are dealing with implicit priors whose densities are unavailable, giving us tools to work with those hard cases.

Tom: It's a powerful tool because it achieves state-of-the-art predictive performance on numerous benchmarks, but we have to remember the reliance on prior quality and the associated computational overhead.

Meng: And for production use, we need to be very careful about setting up the system correctly due to that setup cost and dependency on those prior evaluations.

Lu: It's a significant contribution because it shows how these concepts can be integrated into function-space variational inference through a shared pathwise map.

Jane: So in summary, Generalized Matheron Variational Implicit Processes gives us a way to handle implicit priors by using an empirical Matheron correction anchored at inducing inputs.

Tom: That's the gist of it for now. Next time we'll take another paper and see what they have up their sleeves.

The paper's summary: Tom: So, we're looking at the paper "Generalized Matheron Variational Implicit Processes," and its core idea is using this new method to get posterior samples even when you don't have a direct probability density for the function space.

Jane: That’s right. They introduce this pathwise variational family that uses an empirical Matheron correction anchored at inducing locations, which lets the posterior retain the actual structure and variability of the original implicit process.

Lu: The construction uses a scalar output formula, f(times) = g(times) + Z(times)

mu + D a - g(Z): , where g is just a sample from the prior function distribution.

Meng: That Z part, the empirical operator, they estimate it from an empirical prior-sample covariance. So it’s not just some abstract math; it’s based on actual samples we have.

Tom: Exactly, and that’s what makes it practical because you don't need to know the function space density directly to build that correction term.

Jane: And they show this empirical operator preserves the first two moments of the original implicit process even when the prior isn't Gaussian, which is a big deal for complex real-world data.

Lu: They also prove that for Gaussian priors, this construction exactly recovers standard inducing-variable GP constructions, and it works for any coefficient law q phi(a).

Meng: But they do have some costs. They show the computational cost of building that empirical operator Z is proportional to something like O C bank(N + M, B) + BNM + BM squared + NM squared + M cubed.

Tom: That’s a hefty setup cost, which is why they compare it against simpler coefficient-only variational inference methods.

Jane: And while the setup is expensive, the results are competitive. They show strong empirical performance in regression and classification tasks, achieving low mean RMSE and CRPS on things like Lotka–Volterra forecasting.

Lu: Prior learning seems crucial here; tunable-prior GMVIP consistently outperforms its frozen counterpart on every dataset and metric reported.

Meng: Tuning the prior sounds necessary, but they also flagged that the empirical operator Z is sensitive to how good your prior bank is; poor choices can degrade performance.

Tom: And they also point out a limitation: for non-Gaussian or highly structured processes, this method performs inference under a surrogate prior induced by the replacement map that might differ significantly from the original implicit process.

Jane: So while it’s flexible, you have to be careful because of how much it relies on those initial prior samples you feed into the system.

Lu: They also show that increasing the inducing basis size S generally improves point forecasting accuracy, but they don't guarantee monotone gains across all metrics.

Tom: And there’s this idea about the residual path acting as both an additional source of predictive uncertainty and a structural regularizer against overfitting the finite-dimensional inducing representation.

Jane: That residual path is interesting because removing it leads to increased KL divergence in the inducing coefficient space, suggesting it carries significant information not captured by just those points.

Lu: In conclusion, Generalized Matheron Variational Implicit Processes gives us a way to handle implicit priors by using an empirical Matheron correction anchored at inducing locations and shows how it preserves the structure and variability of the original process.

Meng: It’s a sophisticated framework that gets state-of-the-art predictive performance on many benchmarks, but you gotta remember the reliance on prior quality and that setup cost.

Tom: And for production use, we need to be very mindful of setting up the system correctly due to that dependency on those prior evaluations.

Jane: It's a significant contribution because it shows how these concepts can be integrated into function-space variational inference through a shared pathwise map, giving us tools for those hard cases where densities are unavailable.

The paper's improvements: Tom: So, we've been talking about how they build these posterior samples using that empirical Matheron correction anchored at inducing locations, and now we’re looking at what they suggest to improve the system itself.

Jane: They’re saying they can recover both standard Gaussian Process priors and Sparse Variational GP posteriors exactly when you use the matching operator.

Lu: That's because Proposition three shows that GMVIP reduces exactly to classical Matheron sampling on sparse variational GPs under that condition.

Meng: So for someone trying to deploy a standard GP setup, this means they can use it if they stick to the right inducing setup and matching operator.

Tom: And they also prove that the empirical Matheron operator preserves the first two moments of the original implicit process even when your prior is not Gaussian, which is important because it keeps population structure intact.

Jane: That means you don't lose that structure just because you moved away from simple Gaussian assumptions in your prior.

Lu: They also show that for any coefficient law q phi(a), the induced posterior follows this specific form: q f equals Z p zero of f u q phi of u du.

Meng: It sounds like a way to simplify the posterior structure by focusing on those inducing inputs rather than trying to map out every single function in space.

Jane: That’s a nice connection between the complex implicit process and something we already understand well, the standard GP setup.

Lu: They also point out that they can recover standard inducing-variable GP constructions exactly for Gaussian priors, which gives us a solid baseline to compare against.

Meng: But they do have some costs again. The computational cost of building that empirical operator Z is proportional to something like O C bank(N + M, B) + BNM + BM squared + NM squared + M cubed.

Tom: That’s a hefty setup cost, which is why they compare it against simpler coefficient-only variational inference methods.

Jane: And while the setup is expensive, the results are competitive. They show strong empirical performance in regression and classification tasks on things like Lotka–Volterra forecasting.

Lu: Prior learning seems crucial here; tunable-prior GMVIP consistently outperforms its frozen counterpart on every dataset and metric reported.

Meng: Tuning the prior sounds necessary, but they also flagged that the empirical operator Z is sensitive to how good your prior bank is; poor choices can degrade performance.

Jane: So while it’s flexible, you have to be careful because of how much it relies on those initial prior samples you feed into the system.

Lu: They also show that increasing the inducing basis size S generally improves point forecasting accuracy, but they don't guarantee monotone gains across all metrics.

Meng: And there’s this idea about the residual path acting as both an additional source of predictive uncertainty and a structural regularizer against overfitting the finite-dimensional inducing representation.

Tom: That residual path is interesting because removing it leads to increased KL divergence in the inducing coefficient space, suggesting it carries significant information not captured by just those points.

Jane: So if you want to combat overfitting on things like Boston or Yacht datasets, keeping that residual path seems like a practical way to go.

Meng: It’s a sophisticated framework that gets state-of-the-art predictive performance on many benchmarks, but you gotta remember the reliance on prior quality and that setup cost.

Tom: And for production use, we need to be very mindful of setting up the system correctly due to that dependency on those prior evaluations.

Conclusion: Tom: So to wrap up, we're talking about "Generalized Matheron Variational Implicit Processes," which is this method that uses an empirical Matheron correction anchored at inducing locations to get posterior samples for implicit priors.

Jane: Basically, it’s a way to infer functions when you don't have the full probability density of the function space, and it works even with non-Gaussian priors because it preserves the first two moments of the original process.

Lu: It’s a big idea because for Gaussian priors, it recovers standard inducing-variable GP constructions exactly, and for any coefficient law q phi(a), the induced posterior follows that specific form.

Meng: But remember, there is a cost to this; the computational cost of building that empirical operator Z is proportional to something like O C bank(N + M, B) + BNM + BM squared + NM squared + M cubed.

Tom: That’s a hefty setup cost, which is why they compare it against simpler coefficient-only variational inference methods.

Jane: And while the setup is expensive, the results are competitive; they show strong empirical performance in regression and classification tasks on things like Lotka–Volterra forecasting.

Lu: Prior learning seems crucial here; tunable-prior GMVIP consistently outperforms its frozen counterpart on every dataset and metric reported.

Meng: Tuning the prior sounds necessary, but they also flagged that the empirical operator Z is sensitive to how good your prior bank is; poor choices can degrade performance.

Jane: So while it’s flexible, you have to be careful because of how much it relies on those initial prior samples you feed into the system.

Lu: They also show that increasing the inducing basis size S generally improves point forecasting accuracy, but they don't guarantee monotone gains across all metrics.

Jane: So if you want to combat overfitting on things like Boston or Yacht datasets, keeping that residual path seems like a practical way to go.

Meng: It’s a sophisticated framework that gets state-of-the-art predictive performance on many benchmarks, but you gotta remember the reliance on prior quality and that setup cost.

Tom: And for production use, we need to be very mindful of setting up the system correctly due to that dependency on those prior evaluations.

Luis A. Ortega, Andres R. Masegosa, Thomas D. Nielsen

Aalborg University

cs.LG

Submitted: 2026-10-06

Updated: 2026-10-06

Comments: 36 pages, 10 figures, 19 tables. Submitted for review

Code: https://github.com/Ludvins/ImplicitProcessZoo

License: http://creativecommons.org/licenses/by-sa/4.0/

Importance score: 87/100

The gist: Detailed Research Summary: Generalized Matheron Variational Implicit Processes (GMVIP) The provided text details Generalized Matheron Variational Implicit Processes (GMVIP), a novel method designed

Key concepts

Implicit-process priors: These are priors defined over functions (like a complex shape or signal) where you cannot easily calculate how likely any specific function is. They are often generated by simulators or neural networks, making them hard to work with directly for standard statistical methods.
Pathwise Variational Family: GMVIP uses the same mathematical construction method for both the prior (what it starts with) and the posterior (the result). This consistency allows researchers to use a tractable measure, specifically a coefficient-space Kullback-Leibler divergence, which is easier to calculate than standard function space divergences.
Empirical Matheron Operator: This operator acts as a correction term anchored at specific 'inducing inputs.' It is estimated using samples from the prior's covariance structure. This correction allows the method to adjust the initial prior sample to better fit observed data, preserving important structural information.

Summary

Detailed Research Summary: Generalized Matheron Variational Implicit Processes (GMVIP)

The provided text details Generalized Matheron Variational Implicit Processes (GMVIP), a novel method designed for posterior inference when dealing with implicit-process priors—priors defined over functions whose function-space densities are typically unavailable. GMVIP addresses this challenge by introducing a pathwise variational family for such priors.

Core Methodology and Theoretical Framework

GMVIP operates by constructing posterior samples through a specific scalar-output mechanism anchored at a set of inducing inputs, leveraging prior function samples and an empirical correction term derived from prior samples.

The Fundamental Construction:

The method is defined by the following scalar-output construction:

f(times) = g(times) + Z(times) [mu + D a - g(Z)]

where:

g about p 0 is a sample from the prior function distribution.
Z = (z 1,, z M) represents the inducing locations.
Z(times) is a replacement operator.

Key Mechanisms:

Pathwise Variational Family: GMVIP utilizes the same pathwise construction for both the (surrogate) prior and the variational posterior, differing only in the distribution of whitened inducing coefficients, which allows for a tractable coefficient-space Kullback-Leibler divergence.
Empirical Operator (Z): The operator Z can be estimated from an empirical prior-sample covariance, serving as the closest analogue to Gaussian Process (GP) Matheron conditioning: Z = (times, Z) (Z, Z)-1, where is the empirical covariance.
Modularity: The construction is modular, allowing for the estimation of Z from prior samples and the definition of u(r) from mu + D a(r).

Theoretical Contributions:

The paper establishes several key theoretical properties:

Coherent Samples: GMVIP produces coherent posterior function samples requiring only prior sampling and likelihood evaluation.
KL Reduction: Proposition 2 demonstrates that the augmented KL divergence exactly reduces to a coefficient-space KL divergence, and under a cardinal right-inverse condition, this identity holds for the induced function marginal.
Prior Preservation: Proposition 4 shows that the empirical Matheron operator preserves the first two moments of the original implicit process even when the prior is non-Gaussian.
GP/SVGP Recovery: For Gaussian priors, GMVIP recovers standard inducing-variable GP constructions, and for any coefficient law q phi(a), it shows that the induced posterior is q(f) = Z p 0(fu) q phi(u) du.

Training Procedure

The training involves an iterative loop (Algorithm 1 GMVIP training loop):

Draw a minibatch (X B, Y B).
Draw a path a(r) about q phi(a) and a coherent prior function sample g(r) about p 0(g) at the combined set X B Z.
Compute the augmented term: u(r) from mu + D a(r).
Construct the posterior sample: f(r) B from g(r)(X B) + Z(X B)u(Z).
The loss function is defined as: L bt from NBRtr p psi Y B f(r) B - KL(q phi(a) | N (0, I d a)).
Update the parameters (phi, psi) using an Adam step on-L bt.

Performance and Empirical Results

GMVIP has demonstrated strong empirical performance across various tasks:

Competitive Metrics: It is competitive in both regression and classification experiments. For example, in Lotka–Volterra forecasting with a simulator-defined prior, GMVIP achieved the lowest mean RMSE, CRPS, and mixture Negative Log-Likelihood (NLL) among methods in a set of 20 comparisons. In electricity load forecasting, it exhibited the lowest mean NLL, CRPS, and Coverage Quality Metric (CQM).
Prior Importance: Prior learning is crucial; tunable-prior GMVIP consistently outperforms its frozen counterpart on every dataset and metric reported.
Residual Path Role: The residual path acts as both an additional source of predictive uncertainty and a structural regularizer against overfitting the finite-dimensional inducing representation. Removing it leads to increased KL divergence in the inducing coefficient space, indicating that the residual path carries significant information not captured by the inducing representation alone.
Benchmark Strengths: GMVIP is noted as strongest in specific contexts, such as FashionMNIST LeNet comparison and under a common fixed-learning-rate frozen-CLIP CIFAR-10 protocol. On DynaSent, it provides the strongest overall in-distribution and shifted predictive performance.

Limitations and Complexity

Despite its strengths, GMVIP has identified several limitations:

Prior Dependence: It performs inference under the surrogate prior induced by the replacement map (Z), which may differ significantly from the original implicit-process prior for non-Gaussian or highly structured processes.
Empirical Dependency: The empirical operator Z is sensitive to the quality of the prior bank and requires conditioning of the inducing covariance; poor choices can degrade performance.
Computational Cost: Constructing the empirical operator incurs a cost proportional to O C bank(N + M, B) + BNM + BM squared + NM squared + M cubed. After this setup, one posterior sample costs O C g(N + M) + M squared + NM.
Comparison to VIP: Empirical GMVIP is more computationally expensive than coefficient-only VIP-style methods due to the required prior-bank evaluations and covariance estimation.

Comparative Analysis (Ablation Studies)

Ablation studies highlight trade-offs:

Frozen Prior vs. Tunable Prior: Freezing the prior removes train–test discrepancy but typically leads to underfitting both splits (e.g., frozen VIP shows higher train/test RMSE). The results suggest a sharp trade-off between overfit/overconfident tunable priors and underfit/overly diffuse frozen priors, necessitating a joint interpretation of metrics.
Basis Size (S): Increasing the inducing basis size (S) generally improves point forecasting accuracy but does not guarantee monotone gains across all metrics. GMVIP shows the lowest mean RMSE, NLL, CRPS, and ODE residual among learned configurations with 100 inducing variables and 512 prior samples.
Capacity Sensitivity: The comparison between empirical GMVIP (using inducing values) and SIP (using inducing variables) suggests that the computational overhead of GMVIP is justified by its ability to construct a Matheron operator from prior-bank evaluations.

**In conclusion, GMVIP is a sophisticated variational inference framework that effectively handles implicit-process priors by integrating pathwise constructions with an empirical Matheron correction. While it achieves state-of-the-art predictive performance on numerous benchmarks, researchers must account for its reliance on prior quality and the associated computational overhead.

Improvements for AI systems

Bold header: Posterior Inference for Implicit Processes

The improved system can perform posterior inference for general implicit priors by drawing a coherent prior path and replacing its values at inducing locations through an empirical Matheron-style correction, which allows the posterior to retain the structure and variability of the original implicit process.

Bold header: Exact GP/SVGP Recovery

The system can recover both standard Gaussian Process (GP) priors and Sparse Variational GP (SVGP) posteriors exactly, as Proposition 3 states that GMVIP reduces exactly to classical Matheron sampling on sparse variational GPs when the matching operator is used.

Bold header: Prior Mean and Covariance Preservation

The system can maintain the population structure of implicit priors by ensuring that the empirical Matheron operator preserves the first two moments of the original implicit process when the prior is not Gaussian, as shown in Proposition 4.

Bold header: Robustness to Operator Bank Size

The system's performance can be tuned by increasing the number of prior samples, as Table 15 shows that the larger banks improve sharply for metrics like SW2 and W1, suggesting that B = 2048 provides the best aggregate predictive results.

Bold header: Overfitting Mitigation via Residual Path

The system can combat severe overfitting on datasets like Boston and Yacht by retaining the residual path, as it is shown that the residual path acts both as an additional source of predictive uncertainty and as a structural regularizer against overfitting the finite-dimensional inducing representation.

Abstract

Implicit-process priors specify distributions over functions through sample-forward mechanisms such as Bayesian neural networks and stochastic simulators, but their function-space densities are typically unavailable. We introduce Generalized Matheron Variational Implicit Processes (GMVIP), a pathwise variational family for posterior inference with such priors. For Gaussian-process priors, GMVIP recovers the standard inducing-variable variational GP construction; for general implicit priors, its empirical covariance construction preserves the prior mean and covariance in the population limit. GMVIP constructs posterior samples by drawing a function from the prior and applying a correction anchored at a set of inducing inputs. The effect of this correction away from the inducing inputs is determined directly from prior samples, allowing the posterior to retain the structure and variability of the original implicit process. The (surrogate) prior and variational posterior use the same pathwise construction and differ only in the distribution of whitened inducing coefficients, yielding a tractable coefficient-space Kullback-Leibler divergence. Experiments on regression, classification, and forecasting with simulator-defined and retrieval-conditioned empirical trajectory priors show that GMVIP is broadly competitive with existing methods.

Sources

Flow-Transformed Implicit Processes for Function-Space Variational Inference

Listen

Transcript

Key concepts

Summary

Improvements for AI systems

Abstract

Sources

Related papers