Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective

arXiv:2610.12586 · cs.LG · Submitted 2026-10-08 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: Today's paper: "Realism VS Accuracy".

Jane: The gist Event sequence forecasting can be framed as a conditional generative modeling problem where no method excels at both accuracy and realism,

Tom: First, who's behind it and why it matters.

Title and authors: Tom: So, to recap, this paper sets up a challenge by comparing generative models for event sequences against two key ideas: prediction accuracy and distributional realism. They're testing if you can get a model that is both good at predicting the next thing and good at generating things that actually look like they could happen.

Jane: Exactly. They’re using datasets to show that you can’t have one method that excels at both metrics simultaneously, which points to a fundamental trade-off in how we approach these forecasts.

Lu: The paper is introducing a specific setup for this comparison, defining the problem as forecasting S H+one:H+L given history S one:H, where each event is defined by three things: time, an event type, and some other feature <ref:2610.12586#pg1>. This formalizes what they are trying to model.

Meng: So it’s not just a simple next-step prediction; it's a longer sequence forecast, L steps ahead, which adds a layer of complexity for the models they are testing.

Lalam: They also introduce two specific metrics to measure these things: Temporal Matching Score, or TMS, which measures prediction accuracy based on aligning predicted and true events using per-feature normalization and permutation of horizon positions.

Tom: And then there's the Conditional Detection Fooling Score, or cDFS, which is their measure for distributional realism—basically how hard it is to fool a classifier into thinking a forecast is real.

The paper's summary: Jane: When we look at the actual summary of "Realism VS Accuracy: Event Sequence Forecasting from a Generative Modeling Perspective," they show that they've adapted representatives from three different classes of models to fit this event sequence setting.

Lu: They take a data-space diffusion model, which is CDiff, and a latent diffusion model, which adapts TabSyn, and even a non-diffusion method called DEF. This gives them a broad view of what’s out there in the literature for event sequences.

Meng: It seems like they are trying to cover the spectrum from pure data diffusion to these more complex latent space methods and even standard non-diffusion approaches.

Tom: What's interesting is that they use these different model types on the same problem, which helps isolate whether the trade-off we see is due to the model architecture itself or something else.

Jane: The main finding they highlight is that their evaluation shows no method excels at both TMS and cDFS, clearly demonstrating that there's a fundamental conflict between achieving high prediction accuracy and achieving high distributional realism.

Lalam: It’s like if you try to make the forecast perfectly match the most likely path, you end up missing out on generating sequences that are actually plausible under the underlying data distribution.

The paper's improvements: Tom: The authors introduce Wasserstein BaryBooster, which they call a post-hoc procedure for aggregation. It takes N stochastic samples from any base generator and turns them into a single prediction that is optimal according to TMS.

Jane: So this is the mechanism they propose to bridge that gap; it essentially aggregates the potential of multiple samples from one model into one single forecast that maximizes accuracy under TMS, while still keeping some measure of realism.

Lu: The procedure involves finding an arg min where they use a discrete version of the Wasserstein barycenter as a way to find def that minimizes TMS over all those samples, which is pretty heavy computation at inference time.

Meng: That sounds computationally expensive for production if you actually have to run it every time you need a forecast. How does the complexity scale when we try to deploy this?

Tom: The paper shows that Wasserstein BaryBooster produces the highest TMS across all datasets while still retaining a non-trivial cDFS, which means it’s doing better than just picking one of the base models.

Lalam: They also introduce two new metrics: TMS for accuracy and cDFS for realism, which they argue are necessary because previous work focused almost entirely on prediction accuracy and ignored distributional realism.

Conclusion: Jane: So to wrap up, the main implication of this paper is that we need to stop treating prediction accuracy and distributional realism as things you can just optimize independently.

Tom: The Wasserstein BaryBooster procedure gives us a concrete way to harvest the potential of multiple stochastic samples from a realist model, turning it into a single predictor optimal under TMS.

Lu: It shows that even with different architectures like the LDM or DDM, there’s this clear trade-off plot where high prediction accuracy usually means low distributional realism and vice versa.

Meng: From an engineering standpoint, the main caveat is that implementing Wasserstein BaryBooster introduces substantial overhead because it requires generating multiple candidate trajectories and solving that iterative aggregation procedure at inference time.

Lalam: And the paper’s limitation is that they only studied card transactions on a relatively small number of datasets, so we can't say this works perfectly across every domain yet.

Dmitry Osin, Egor Surkov, Petr Mokrov, Igor Udovichenko, Ivan Karpukhin, Evgeny Burnaev, Egor Shvetsov, Alexander Korotin

Applied AI Institute 2 Applied AI Institute 3 Vega Institute Foundation 4 SB AI Lab 5 AXXX

cs.LG

Submitted: 2026-10-08

Updated: 2026-10-08

Comments: Spotlight at the NeurIPS 2026 Track on Evaluations and Datasets. Code: https://github.com/xaosina/event-sequence-forecasting

Code: https://github.com/xaosina/event-sequence-forecasting

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 84/100

The gist: The gist Event sequence forecasting can be framed as a conditional generative modeling problem where no method excels at both accuracy and realism, highlighting a fundamental trade-off between

Key concepts

Event Sequence Forecasting: This is the task of predicting a future chain of events (like transactions or medical readings) given a history. Each event in the sequence has different types of data, including numbers and categories. The goal is to forecast many future steps ahead based on past observations.
Distributional Realism: This measures how realistic the predicted sequence looks, focusing on the underlying probability distribution of possible events rather than just a single best guess. High realism means the model captures the true variety and structure of real-world event patterns.
Prediction Accuracy (TMS): This metric evaluates how close a forecast is to the actual observed sequence by comparing them. It uses normalization and permutation techniques to measure matching across features and time steps, aiming for high precision in predicting the exact next event.
Wasserstein BaryBooster: This is a novel algorithm that takes multiple random samples from a diffusion model (a type of generative model) and aggregates them into one optimized forecast. It uses a discrete version of the Wasserstein barycenter to find the best prediction that balances high accuracy with maintaining some level of distributional realism.

Terminology

Accuracy — 1× in this paper · explained in 1 episode
Generative Modeling — 1× in this paper · explained in 3 episodes

Summary

The gist Event sequence forecasting can be framed as a conditional generative modeling problem where no method excels at both accuracy and realism, highlighting a fundamental trade-off between prediction accuracy and distributional realism The paper introduces Wasserstein BaryBooster, an algorithm that aggregates samples from a realism strong diffusion model into a state-of-the-art predictor, demonstrating that distributional realism can be converted into prediction accuracy.

Problem Statement and Related Work

The problem is formalized as forecasting the sequence of future events SH+1:H+L given a history S1:H, where each event is a 3-tuple (ti, xi, ki). The work positions itself among three relevant bodies of work: generative models applicable to event sequence data (§2.1), existing benchmarks in the space (§2.2), and metrics previously used to evaluate such models (§2.3).

Generative Models for Event Data

The paper characterizes methods by their model type, handling of sequential data, and feature types they can generate. It fills a gap by adapting representatives of all three classes—a data-space diffusion model, a latent diffusion model, and a non-diffusion method—to the general event-sequence forecasting setting. The methods studied include CDiff [Zeng et al., 2024], TabSyn [Zhang et al., 2024], and DEF [Karpukhin and Savchenko, 2026]

Benchmarks for Event Sequences

The benchmark is designed to probe the trade-off by evaluating both prediction accuracy and distributional realism. The authors present a benchmark comprising several large-scale event sequence datasets and a set of metrics that capture both perspectives on forecast quality, including cDFS (Conditional Detection Fooling Score), an adaptation of the classifier two-sample test [Lopez-Paz and Oquab, 2017], and TMS (Temporal Matching Score). The paper finds that no method excels at both TMS and cDFS, exposing a fundamental trade-off between prediction accuracy and distributional realism.

Metrics for Generative Sequence Models

The paper introduces two novel metrics to bridge the gap between prediction accuracy and distributional realism. Prediction accuracy is measured by Temporal Matching Score (TMS), which uses per-feature, baseline-rescaled normalization and a one-to-one permutation of horizon positions instead of hand-tuned costs. Distributional realism is measured by Conditional Detection Fooling Score (cDFS), a conditional variant of the classifier twosample test [Lopez-Paz and Oquab, 2017].

Models for Evaluation

The evaluation involves representative methods from three major classes: a latent diffusion model (LDM), a data diffusion model (DDM), and a state-of-the-art non-diffusion method (DEF). The LDM adapts TabSyn [Zhang et al., 2024] to the event-sequence setting, using a β-VAE to encode features into latent space and Gaussian diffusion over sequences of latent event representations. The DDM uses CDiff [Zeng et al., 2024], combining Gaussian diffusion for inter-arrival times and uniform categorical diffusion for each categorical feature.

Wasserstein BaryBooster

This post-hoc procedure turns N stochastic samples from any base generator into a single continuation optimal under TMS. The optimization procedure is given in Appendix F, where the discrete TMS analog of a Wasserstein barycenter is used to find Sˆ⋆ def = arg min Sˆ X N n=1 dTMS(S, ˆ Sˆn). This procedure produces the highest TMS across all datasets while still retaining a non-trivial cDFS.

Experiments and Results

The experiments forecast at fixed horizons L ∈ L = 4, 16, 32, and 64. The main results use L=32 (Table 3), showing a sharp split between the two metrics where LDM is the only generative model to achieve a non-trivial cDFS. Wasserstein BaryBooster yields the highest TMS across all datasets while still retaining a non-trivial cDFS. The trade-off plot shows that methods with high prediction accuracy show low distributional realism and vice versa.

Limitations and Broader Impacts

The study uses a relatively small number of datasets and focuses narrowly on card transactions, noting that rich corpora are rarely released publicly. The Wasserstein BaryBooster introduces substantial computational overhead since it requires generating multiple candidate trajectories and solving an iterative barycentric aggregation procedure at inference time. The forecasts studied in this paper can support planning and risk analysis by exploring different paths a client’s future may take.

Reproducibility

The full source code, with documentation, is publicly available (§4) and enables reproduction of the experiments. All neural models are implemented in PyTorch, and training and evaluation follow a single shared protocol.

Open Access to Data and Code

The full source code, with documentation, is publicly available (§4) and enables reproduction of the experiments. The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data.

Experiment Statistical Significance

The authors address statistical significance in Appendix E.5. They report mean and standard deviations over three random seeds in Table 10 for the spotlight methods. The authors draw their conclusions on metric gaps that are much larger than standard deviations across runs, so they do not question the statistical significance.

Experiment Compute Resources

Appendix I.3 states single NVIDIA A100 training; Appendix J tabulates train/inference wall times and GPU memory per model/dataset while noting further compute for grids and screening (Appendix I.3).

Code of Ethics

The research conducted in the paper conforms, in every respect, with the NeurIPS Code of Ethics. The authors study evaluation on curated public transactional benchmarks rather than scraped media; no human-subjects experimentation was conducted.

Broader Impacts

The forecasts studied in this paper can support planning and risk analysis by exploring different paths a client’s future may take. Analysts can explore the different paths a client’s future may take, rather than a single average forecast. The metrics, benchmark, and methods are not specific to transactions and can bring these benefits to event sequences in other domains such as healthcare and e-commerce.

Safeguards

The authors study evaluation on curated public transactional benchmarks rather than scraped media; no gated-release safeguards apply. The released code exposes methodology but not frontier-scale multimodal generators, so no gated-release safeguards apply.

Licenses for Existing Assets

Yes. All datasets used have accompanying URLs and are explicitly cited; all codebases and models employed are properly licensed, and their licenses are respected and listed in the repository README. The name of the license (e.g., CC-BY 4.0) should be included for each asset.

New Assets

Benchmark metrics and BaryBooster are defined in the main text and Appendices C–F; consolidated training documentation appears in Appendix I with the public code repository referenced in §4. The authors should communicate the details of the dataset/code/model as part of their submissions via structured templates.

Crowdsourcing and Research with Human Subjects

No crowdsourcing or primary human-subjects studies were conducted.

Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects

No IRB-governed human-subjects component; only secondary use of public datasets.

Declaration of LLM Usage

Large language models are not part of the proposed forecasting methods; any writing assistance is incidental to the technical contribution. The core method development in this research does not involve LLMs as any important, original, or non-standard components.

References

Alexandra Bazarova, Maria Kovaleva, Ilya Kuleshov, Evgenia Romanenkova, Alexander Stepikin, Aleksandr Yugay, Dzhambulat Mollaev, Ivan Kireev, Andrey Savchenko and Alexey Zaytsev. Learning transactions representations for information management in banks: Mastering local, global and external knowledge. International Journal of Information Management Data Insights 5(1):100323, 2025.

Alistair Johnson, Lucas Bulgarelli, Tom Pollard, Brian Gow, Benjamin Moody, Steven Horng, Leo Anthony Celi and Roger Mark. MIMIC-IV PhysioNet October 2024 doi: 10.

Improvements for AI systems

Weighted aggregation for prediction accuracy using Wasserstein BaryBooster can be applied to any base generator to produce SoTA prediction accuracy by finding a single continuation optimal under TMS, converting a generative model into a competitive predictor.
The system can generate sequences that are both accurate and realistic by leveraging the trade-off analysis, as shown in Figure 2, where methods like LDM-WB achieve high TMS while retaining a non-trivial cDFS.
A novel forecasting system can be developed that utilizes the Temporal Matching Score (TMS) to measure accuracy by aligning predicted and true events via a minimum-cost edit over insertions, deletions, and event moves, even with heterogeneous features.
The system can provide a holistic quality assessment by combining prediction accuracy with distributional realism using the metrics cDFS and TMS, as the paper finds that no method excels at both TMS and cDFS, exposing a fundamental trade-off.
A robust event sequence predictor can be built by employing Wasserstein BaryBooster to harvest the potential of multiple stochastic samples, resulting in a single continuation optimal under TMS.

Abstract

Event sequences are chains of random events, each described by heterogeneous (numerical and categorical) features. The problem of event sequence forecasting occurs in finance, healthcare, business process intelligence, and other fields. In this work, we study multi-horizon forecasting of event sequences as a conditional generative modeling problem. We evaluate forecasts not only for prediction accuracy, but also for distributional realism. To this end, we present a benchmark comprising metrics for both quality aspects, strong statistical baselines, recent models adapted from adjacent fields, and curated real-world datasets. Careful evaluation reveals that no method excels at both accuracy and realism, highlighting a fundamental trade-off between the two. Finally, we introduce Wasserstein BaryBooster, an algorithm that aggregates samples from a realism-strong diffusion model into a state-of-the-art predictor, demonstrating that distributional realism can be converted into prediction accuracy. Code is available at https://github.com/xaosina/event-sequence-forecasting.