DiTS: Multimodal Diffusion Transformers Are Time Series Forecasters

The gist: DiTS proposes Diffusion Transformers for Time Series, a Multimodal Diffusion Transformer that treats endogenous targets and exogenous covariates as distinct modalities to jointly condition future generation on target history and available covariates

DiTS Architecture

DiTS models endogenous targets and exogenous covariates as distinct modalities, exploiting their temporal alignment for fine-grained conditional control Generation is jointly conditioned on target history and available covariates. The model treats endogenous targets and exogenous covariates as distinct modalities with different sources, semantics, and statistical characteristics but shared temporal coordinates. Target patches retain historical context, while a Covariate Encoder preserves patch-wise external information.

Time-aligned Modulation

DiTS introduces Time-aligned Modulation, which extends AdaLN with the temporal-alignment prior to generate patch-wise modulation parameters from aligned covariates and diffusion time. This mechanism connects target representations and covariate representations, where temporal position determines where conditioning applies, while diffusion time specifies the generation stage. The patch-wise modulation parameters are generated by applying an MLP to a sinusoidal encoding of the diffusion time combined with the aligned covariate conditions.

Flow Matching and Velocity Prediction

The model adopts conditional flow matching, specifically rectified flow, to learn a continuous transformation from Gaussian noise to the data distribution. The training objective minimizes the mean squared error between the predicted velocity and the target drift. This learning of a network velocity field is conditioned on target history and exogenous covariates, allowing covariate-dependent distributional information to be relevant to velocity prediction even when the original conditional mean is unchanged.

Tokenization and Conditioning

The input representation involves Patch Embedding for both target history and future state, followed by a Covariate Encoder that encodes covariates separately while preserving temporal correspondence with target patches. The Covariate Encoder preserves patch-wise external information, yielding a sequence of covariate representations where each temporal patch has a separate condition.

Experimental Results

Experiments demonstrate strong forecasting performance across diverse covariate-aware forecasting tasks, achieving leading aggregate results in both deterministic and probabilistic forecasting. DiTS achieves the lowest geometric mean WQL and MASE in the reported comparison for both deterministic and probabilistic evaluations. Time-aligned Modulation is shown to achieve the lowest average MSE/MAE among conditioning mechanisms without a separate evolving covariate stream. DiTS achieves favorable quality with few sampling steps, balancing accuracy and generation cost.

Model Efficiency

DiTS surpasses baselines in WQL with three steps for generation latency. Its training memory lies below TimeGrad and TempFlow but above DeepAR and GCGNet. DiTS combines accurate forecasting with efficient few-step generation, balancing predictive performance and computational cost.

Sampling and Noise Scheduling

Training-time sampling is performed using a logit-normal distribution for training times, which yields the lowest mean MSE among the selected configurations. The inference budget utilizes five Euler integration steps per trajectory and averages ten independently initialized forecast samples for point prediction. Rectified flow is found to yield the lowest five-dataset average MSE and MASE under this setup.

Prediction Parameterization

The v-prediction/v-loss configuration yields the lowest five-dataset average MSE and MAE among the four evaluated parameterizations. This suggests that the preferred parameterization may depend on the data modality. Classifier-free guidance sensitivity shows that guidance scale 1, which does not amplify the conditional–unconditional difference, gives the most consistent results.

Conclusion

These findings support time-aligned conditional generation for covariate-aware forecasting. DiTS achieves favorable quality with few sampling steps, balancing accuracy and generation cost. The paper supports modeling covariate-dependent distributions while retaining accurate point forecasts, consistent with richer covariate supervision through noisy-state conditioning.

References

Juan Miguel Lopez Alcaraz and Nils Strodthoff. Diffusion-based time series imputation and forecasting with structured state space models. arXiv preprint arXiv:2208.09399, 2022

Defu Cao, Wen Ye, Yizhou Zhang, and Yan Liu. TimeDiT: General-purpose diffusion transformers for time series foundation model. arXiv preprint arXiv:2409.02322, 2024

Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00426, 2023

Shoufa Chen, Chongjian Ge, Yuqi Zhang, Yida Zhang, Fengda Zhu, Hao Yang, Hongxiang Hao, Hui Wu, Zhichao Lai, Yifei Hu, et al. Goku: Flow based video generative foundation models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2025

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning 2024

Shibo Feng, Chunyan Miao, Zhong Zhang, and Peilin Zhao. Latent diffusion transformer for probabilistic time series forecasting. In Proceedings of the AAAI Conference on Artificial Intelligence volume 38 pp. 11979–11987 2024

Rob J Hyndman and George Athanasopoulos. Forecasting: principles and practice. OTexts, 2018

Rob J Hyndman and Yeasmin Khandakar. Automatic time series forecasting: the forecast package for r Journal of statistical software, 27:1–22, 2008

Jesus Lago, Grzegorz Marcjasz, Bart De Schutter, and Rafał Weron. Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark. Applied Energy 293:116983, 2021

Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise. arXiv preprint arXiv:2511.13720, 2025 URL https://arxiv.org/abs/2511.13720

Zhengyu Li, Xiangfei Qiu, Yuhan Zhu, Xingjian Wu, Jilin Hu, Chenjuan Guo, and Bin Yang. DAG: A dual correlation network for time series forecasting with exogenous variables. arXiv preprint arXiv:2509.14933v3 2026 URL https://proceedings.iclr.cc/paper files/paper/2026/hash/

Bryan Lim, Sercan Ö Arık, Nicolas Loeff, and Tomas Pfister. Temporal fusion transformers for interpretable multi-horizon time series forecasting. International journal of forecasting 37(4): 1748–1764, 2021

Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling. arXiv preprint arXiv:2210.02747 2022

Xingchao Liu, Chengyue Gong, and Qiang Liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. arXiv preprint arXiv:2209.

1. No longer relying solely on direct mean regression for covariate impact; DiTS utilizes conditional flow matching to learn a velocity field conditioned on target history and observed covariates, which allows it to model covariate-dependent distributional differences beyond just the conditional mean.

2. Integration of heterogeneous information; DiTS treats endogenous targets and exogenous covariates as distinct modalities with different semantics, yet leverages their shared temporal coordinates via Time-aligned Modulation to generate patch-wise parameters, ensuring that the model respects the distinct characteristics of each data source while modeling their joint dependencies.

3. Fine-grained conditional control without expanding attention sequence; The introduction of Time-aligned Modulation enables patch-wise modulation parameters from aligned covariates and diffusion time, allowing for fine-grained conditional control without expanding the attention sequence.

4. Enhanced temporal dependency modeling; Temporal attention propagates conditioned information across positions, which allows the model to accommodate lagged dependencies without adding covariate tokens to attention.

5. Probabilistic forecasting with high efficiency; DiTS achieves superior probabilistic performance by using Flow Matching and demonstrates efficiency by achieving favorable quality with few sampling steps, balancing accuracy and generation cost.

While generative modeling facilitates probabilistic time series forecasting, incorporating heterogeneous exogenous information remains challenging. Diffusion Transformers (DiT) provide a scalable framework for conditional generation, yet their adaptation to forecasting calls for conditioning mechanisms tailored to time series. Endogenous targets and exogenous covariates differ in sources, semantics, and statistical characteristics, while sharing temporal coordinates that support fine-grained conditional guidance. Covariates can describe future variability and temporal dependence beyond the conditional mean targeted by direct regression. Motivated by these considerations, we propose Diffusion Transformers for Time Series (DiTS), a Multimodal Diffusion Transformer for covariate-aware forecasting. DiTS models endogenous targets and exogenous covariates as distinct modalities, jointly conditioning future generation on target history and available covariates. Flow matching makes covariate-dependent distributional information relevant to velocity prediction conditioned on noisy future states. We introduce Time-aligned Modulation, extending AdaLN with the temporal-alignment prior to generate patch-wise modulation parameters from aligned covariates and diffusion time. Across diverse covariate-aware forecasting tasks, DiTS achieves strong performance in both deterministic and probabilistic forecasting, demonstrating the effectiveness of conditional generation for both point and distributional forecasting.