Digital Typhoon Dataset V3: Benchmarking for Multi-spectral Extension and Analyzing Systematic Bias in Intensity Regression

arXiv:2610.21654 · cs.CV · Submitted 2026-10-15 · Read on arXiv

Listen

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Digital Typhoon Dataset V3".

Jane: The gist The Digital Typhoon Dataset V3, a new version of the longest typhoon satellite image dataset spanning nearly 50 years,

Tom: First, who's behind it and why it matters.

Paper summary: Tom: So we're looking at this paper called "Digital Typhoon Dataset V3: Benchmarking for Multi-spectral Extension and Analyzing Systematic Bias in Intensity Regression". Basically, they’ve put together a new version of this typhoon satellite image dataset that spans nearly fifty years, and the whole point is to see how well machine learning models perform when they look at this kind of long-term spatio-temporal data <ref:2610.21654#pg1,long-term spatio-temporal data>.

Jane: That sounds like a big undertaking. What’s the main thing they claim about this new version, the Digital Typhoon Dataset V3?

Tom: The major update is adding multispectral satellite data, which means they're enabling multi-channel models instead of just single-channel ones, which is a big step for what we can train. They also looked at how to split the data and did some benchmarking across both the Northern and Southern Hemispheres.

Lu: What I find interesting is their finding that under the current setup, those multi-channel inputs don't actually clearly outperform single-channel inputs when it comes to pressure regression, which is a key metric here.

Meng: That makes sense from a deployment standpoint; if we’re trying to keep things simple and fast for production, a clear advantage isn't always the goal if the performance gain is negligible. What about their findings on specific model choices?

Tom: They looked at ResNet-eighteen versus ResNet-fifty different image types, and various spectral channels, and they found that for pressure regression on the WP dataset, ResNet-eighteen with full-resolution images and just Channel one did the best single-channel performance.

Jane: Okay so it seems that even with all those new spectral channels added in Version three if you're only using one channel for pressure prediction, Channel one is still the strongest choice according to these results <ref:2610.21654#pg1>.

Lu: Yeah, and they also noted that the WP and AU datasets show different performance based on what you're trying to predict; for pressure regression, WP consistently gave lower root mean square error values than AU across all channels and image types.

Tom: That asymmetry is important because it shows you can't just use a single benchmark number without looking at the specific dataset characteristics. And they also found that on the AU dataset, wind regression was where it substantially outperformed the WP data.

Meng: So task dependence is a major factor in how we interpret these benchmarks, which means we have to be careful when applying these numbers to our own production scenarios. What about that systematic bias they mentioned regarding intensity regression?

Jane: Oh, they analyzed some systematic bias where the models tend to underestimate the intensity of strong typhoons and overestimate the weak ones. That’s a real problem if you're trying to predict severe weather events.

Tom: They showed this bias is especially large for pressures below nine hundred twenty hPa, where the mean bias exceeds ten hPa in the most extreme cases. And they tested some imbalance-aware weighting strategies, like inverse-density weighting inspired by BalancedMSE, but those didn't actually remove that bias.

Lu: That suggests the bias is more fundamental than just a simple data distribution issue; you need something deeper to fix it. They showed that even with these advanced weightings, they couldn't get the bias under control for strong typhoons.

Jane: It sounds like a warning about how tricky intensity regression can be when dealing with rare, extreme events in the data. So what’s the overall conclusion from this paper on Digital Typhoon Dataset V3?

Tom: The main point is that under their experimental setup, Channel one paired with full-resolution images and ResNet-eighteen remains the strongest single-channel configuration for pressure regression.

Jane: And they also concluded that adding more spectral information in the multi-channel dataset didn't provide a clear performance advantage for pressure regression.

Lu: They summarized it by pointing out that Channel one along with the similar infrared Channel two contains more relevant information than the other channels for pressure prediction <ref:2610.21654#pg1>.

Meng: So, if we're building a system, this paper suggests that maybe focusing on optimizing those specific core channels might be more efficient than trying to use every single spectral channel available in V3.

Tom: Exactly. And they also pointed out a limitation: to effectively use multi-channel models, you actually need to reduce the dataset size because multiple channels aren't always available in practice.

Jane: That’s a practical constraint we have to consider when designing systems for real-world deployment, especially when dealing with these complex datasets. So what are the next steps they suggest?

Lu: Future research should focus on using existing data more effectively and developing models that are better suited to the characteristics of those extreme events.

Meng: I agree, and another direction is looking into alternative learning strategies for imbalanced regression, like improved formulations of density-aware loss functions or adaptive weighting schemes.

Tom: And prediction calibration or uncertainty-aware approaches could provide more reliable estimates specifically for the rarest and most intense typhoons. That sounds like a necessary evolution in the modeling approach.

Jane: It seems they’re pushing toward methods that are specifically tuned to handle the imbalance and extremity of typhoon data, which is where things get really interesting for predicting severe weather.

Conclusion: Tom: So we're wrapping up this look at the Digital Typhoon Dataset V3, which is basically checking how these long-term typhoon satellite images hold up when you try to use them for machine learning, and also digging into some weird bias issues they found with intensity predictions.

Jane: It sounds like the authors are trying to figure out if adding all those extra spectral channels actually helps the models get better at predicting things, or if there's just noise in the system.

Tom: Exactly. The title itself says they are benchmarking for multi-spectral extension and analyzing systematic bias in intensity regression. It tells you they're doing two main things: testing if more colors help, and looking for those weird errors when we try to predict how strong a typhoon will be.

Lu: From what I’m seeing in the paper, the authors are focusing on how single-channel performance compares to multi-channel performance for pressure regression—that's one part of the benchmark they did.

Meng: And they found that adding multiple spectral channels didn't really give a clear advantage over just using one channel for pressure prediction under these specific settings. That means if you’re deploying something, maybe you don’t need to worry about feeding in every single color at once.

Jane: So, the main conclusion here is that while Version three has more data and more channels, it didn't immediately give a clear performance boost for pressure prediction compared to the simpler setups.

Tom: Right. And they did point out a big limitation too—that to really use those multi-channel models effectively, you might actually need to reduce the dataset size because not every single channel is always available in real applications.

Lu: That’s a practical constraint on using these datasets in the field, I agree. And they also spent time looking at intensity regression bias, which they found systematically causes the models to underestimate strong typhoons and overestimate weak ones.

Jane: So, even with this massive dataset, we still have that fundamental issue where predicting an extreme event like a super intense typhoon is actually harder for the AI model to get right.

Tom: That’s what they were showing—that even after all this data collection and model training, there's still a real problem with handling those rare, intense events accurately.

Lalam: I see that the authors tested some weighting strategies to try and fix that intensity bias, but those methods weren't enough to remove the error under their testing conditions.

Lu: So it confirms what they were suggesting—that we need better loss functions or uncertainty-aware approaches specifically for these kinds of imbalanced regression tasks.

Jane: It shifts the focus from just getting a good pressure prediction number to making sure our AI systems are robust when dealing with those truly extreme weather scenarios.

Asanobu Kitamoto, Victor Le Louarn, Noelia Falczuk, Ngoc Anh Tong, Anja Veleva

National Institute of Informatics · Typhoon Science and Technology Research Center, Yokohama National University, Japan · University of Artois, France · University of Buenos Aires, Argentina · Hanoi University of Science and Technology, Vietnam · RWTH Aachen University, Germany

cs.CV

Submitted: 2026-10-15

Updated: 2026-10-15

Comments: 23 pages, 6 figures, dataset paper

Code: https://github.com/kitamoto-lab/digital-typhoon

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 77/100

The gist: The gist The Digital Typhoon Dataset V3, a new version of the longest typhoon satellite image dataset spanning nearly 50 years, was designed to benchmark machine learning models on long-term

Key concepts

Multi-channel Models: These are machine learning models that use data from several different spectral bands (like infrared or water vapor channels) simultaneously. The goal is to see if combining more types of satellite data helps the model predict typhoon characteristics more accurately than using only one type of data.
Pressure Regression: This is a specific task where the machine learning model tries to predict the central atmospheric pressure associated with a typhoon. The study compared how well different models and input data (like image resolution or spectral channels) perform at this specific prediction task.
Systematic Bias in Intensity Regression: This refers to a consistent error pattern where the model systematically misses the true intensity of certain events, specifically strong typhoons. The research showed that models tend to underestimate the pressure for very intense typhoons, which is a critical issue for accurate forecasting.
Inverse-Density Weighting Strategy: This is a method used to adjust how much importance is given to different data points during training. It's inspired by techniques like BalancedMSE and aims to reduce bias by giving more weight to underrepresented, extreme events, such as very strong typhoons.

Terminology

Benchmarking — 1× in this paper · explained in 4 episodes
Systematic Bias — 1× in this paper · explained in 3 episodes

Summary

The gist The Digital Typhoon Dataset V3, a new version of the longest typhoon satellite image dataset spanning nearly 50 years, was designed to benchmark machine learning models on long-term spatio-temporal data and investigate systematic bias in intensity regression.

Dataset Extensions and Characteristics

The Digital Typhoon Dataset V3 extends the original dataset by incorporating multispectral satellite data, enabling multi-channel models<ref:2610.21654#pg8> The latest addition to Dataset V3 is multispectral satellite data, enabling multi-channel models<ref:2610.21654#pg8>. Version 3 added a second infrared channel, a water vapor channel, and a near-infrared channel to capture different aspects of tropical cyclones<ref:2610.21654#pg4> The first satellite, Himawari-1, has two channels: infrared and visible<ref:2610.21654#pg4>. Channel 3, called Water Vapor, was introduced after GOES-9 and captures moisture in the upper troposphere<ref:2610.21654#pg4> Channel 4, called Near Infrared, spans the electromagnetic spectrum that overlaps the visible and infrared regions<ref:2610.21654#pg4>. Users may wonder why we did not include visible channels in the dataset, as they have been observed throughout the Himawari series<ref:2610.21654#pg4> We decided not to include visible channels for the following reasons<ref:2610.21654#pg4> First, because visible channels measure sunlight reflectance, they are meaningful only during the day<ref:2610.21654#pg4> Second, visible channels have historically been more susceptible to noise; over extended periods, they become significantly contaminated and therefore unusable for quantitative analysis<ref:2610.21654#pg4>.

Benchmarking Single-Channel Models

The benchmarking experiments compare base model architectures between ResNet-18 and ResNet-50, image types, and spectral channels<ref:2610.21654#pg8> We first assess single-channel models and compare their performance across channels, image types, and model architectures<ref:2610.21654#pg8>. For the pressure regression task on the WP dataset, ResNet-18 yields the best results<ref:2610.21654#pg10> The lowest error is observed for Channel 1 using Full images, where the model achieves an RMSE of 6.56 ± 0.06 hPa<ref:2610.21654#pg10>. Regarding image types, Full images yield the best results in most cases<ref:2610.21654#pg10>.

Hemispheric Differences and Task Asymmetry

A comparison between the WP and AU datasets reveals task-dependent performance asymmetries<ref:2610.21654#pg10> For pressure regression, WP consistently achieves lower RMSE values than AU across the evaluated channels and image types<ref:2610.21654#pg10>. In contrast, AU substantially outperforms WP on the wind regression task<ref:2610.21654#pg10>. This asymmetry stems from several factors across the two hemispheres, including dataset size and different operational procedures in regional meteorological agencies<ref:2610.21654#pg10>

Multi-channel Model Performance

When comparing single-channel and multi-channel models on the pressure regression task, adding multiple spectral channels does not clearly improve the best single-channel model for pressure regression<ref:2610.21654#pg10> The best multi-channel models achieve an RMSE of 6.8308 hPa on WP (Channel 1 + 2 + 3), which is 0.10 hPa lower than the Channel 1 model on the same dataset (6.9270 hPa)<ref:2610.21654#pg10>. Overall, the results indicate comparable performance rather than a clear advantage of multi-channel inputs<ref:2610.21654#pg10>.

Systematic Bias in Intensity Regression

The study analyzed systematic bias in intensity regression, which leads to underestimation of intensity for strong typhoons<ref:2610.21654#pg8> We find a systematic bias in intensity regression, as shown in Figure 3<ref:2610.21654#pg9>. The model underestimates strong typhoons and overestimates weak ones<ref:2610.21654#pg9> Errors are especially large for pressures below 920 hPa across all splits, with the mean bias (blue curve) exceeding 10 hPa in the most extreme cases<ref:2610.21654#pg9>.

Bias Mitigation Strategies

To address systematic bias for strong typhoons, we explore two strategies in Section 5.2 and Section 5.3: sample weighting to account for the underrepresentation of extreme events, and an inversedensity weighting strategy inspired by BalancedMSE<ref:2610.21654#pg9>. The inverse-density formulation reduces bias magnitudes at low pressures<ref:2610.21654#pg9> This result is consistent with BalancedMSE’s motivation: reducing the influence of a highly imbalanced target distribution in regression<ref:2610.21654#pg9>.

Overall Conclusion

Under the current experimental setting, Channel 1 with Full-resolution images and ResNet-18 provides the strongest and most consistent single-channel configuration for pressure regression, while the additional spectral information available in the multi-channel dataset does not translate into a clear performance advantage<ref:2610.21654#pg9>. The results can be summarized as follows: 1. Channel 1, together with the similar infrared Channel 2, contains more relevant information than the other channels<ref:2610.21654#pg9>. 3. To use multi-channel models, we need to reduce the dataset size because multiple channels are not always available<ref:2610.21654#pg9>. 4. Overall conclusion: Under the current experimental setting, Channel 1 with Full-resolution images and ResNet-18 provides the strongest and most consistent single-channel configuration for pressure regression, while the additional spectral information available in the multi-channel dataset does not translate into a clear performance advantage<ref:2610.21654#pg9>. The limitation is particularly relevant for strong typhoons, which are already under-represented in the data<ref:2610.21654#pg9>.

Future Research Directions

Future research should focus on using existing data more effectively and developing models better suited to the characteristics of extreme events<ref:2610.21654#pg9> One promising direction is to investigate alternative learning strategies for imbalanced regression, including improved formulations of density-aware loss functions and adaptive weighting schemes<ref:2610.21654#pg9>. Another direction is to investigate prediction calibration or uncertainty-aware approaches, which could provide more reliable estimates for the rarest and most intense typhoons<ref:2610.21654#pg9>. Future work could also explore using additional information already associated with the observations, such as temporal context or complementary meteorological variables<ref:2610.21654#pg9>.

Data Correction Impact

The correction described in Section 2.5 changes only a small fraction of the training images, which are blank or mostly missing in V3-2025.R1 and paired with valid labels<ref:2610.21654#pg9>. The corrupted images strongly affect the training error: with 0.3–0.9% corrupted training observations, the training RMSE at epoch 50 is 2.0–2.9 hPa on V3-2025.R1 and 1.3 hPa on V3-2025.R2<ref:2610.21654#pg9>. The effect on the evaluation RMSE is much smaller<ref:2610.21654#pg9>. The benchmarks of Section 4, computed on V3-2025.R1, therefore remain representative for V3-2025.R2<ref:2610.21654#pg9>.

Optimizer Selection

We replaced SGD with the Adam optimizer, which combines adaptive learning rates with momentum-based updates to improve convergence<ref:2610.21654#pg10> Based on these findings, we performed all subsequent experiments in this paper using Adam<ref:2610.21654#pg10>. Using a single optimization strategy lets the remainder of the study focus on the effects of the proposed architectures and experimental configurations while minimizing variability introduced by the optimization process<ref:2610.21654#pg10>.

Data Splitting Protocol

We primarily use the SS-L2 split for the Digital Typhoon Dataset V3<ref:2610.

Improvements for AI systems

Improved intensity regression for strong typhoons by employing sample weighting to account for underrepresentation of extreme events, specifically using inversedensity weighting strategy inspired by BalancedMSE which is shown to reduce bias magnitudes at low pressures.
Enhanced model robustness against training data corruption by incorporating a mechanism where the corrupted images strongly affect the training error, yet the benchmarks remain representative because the effect on the evaluation RMSE is much smaller (on the order of 0.1 hPa).
Optimized model architecture selection for pressure regression by prioritizing ResNet-18 over ResNet-50 when evaluating single-channel performance, as ResNet-18 generally provides better, more stable performance than ResNet-50 across the evaluated configurations.

Abstract

This paper presents the Digital Typhoon Dataset V3, a new version of the longest typhoon satellite image dataset spanning nearly 50 years, designed to benchmark machine learning models on long-term spatio-temporal data. The latest addition to Dataset V3 is multispectral satellite data, enabling multi-channel models. We investigate data-splitting strategies and present benchmarking results for the Northern and Southern Hemispheres. Experimental results show that, under the current dataset construction and experimental setting, multi-channel inputs do not clearly outperform single-channel inputs for pressure regression. Among the evaluated model architectures, image types, and spectral channels, ResNet-18, full-resolution images, and Channel 1 generally provide the best single-channel performance for pressure regression. Performance also differs between the WP and AU datasets by regression target, with lower RMSE on WP for pressure regression and on AU for wind regression. These results highlight the importance of considering dataset-specific and task-specific characteristics when interpreting benchmark performance. Additionally, we analyzed systematic bias in intensity regression, which leads to underestimation of intensity for strong typhoons. We tested imbalance-aware weighting strategies, including inverse-density weighting inspired by BalancedMSE, and showed that these approaches are not sufficient to remove the bias. The dataset is publicly available at http://agora.ex.nii.ac.jp/digital-typhoon/dataset/ and https://github.com/kitamoto-lab/digital-typhoon/.