A Multi-scale Spatiotemporal Correlation Attention and State Space Modeling-based Approach for Precipitation Nowcasting

ZHENG Hui; CHEN Fu; HE Shuping; QIU Xuexing; ZHU Hongfang; WANG Shaohua

doi:10.11999/JEIT250786

Article Contents

Article Navigation > Journal of Electronics & Information Technology > 2025 >

ZHENG Hui, CHEN Fu, HE Shuping, QIU Xuexing, ZHU Hongfang, WANG Shaohua. A Multi-scale Spatiotemporal Correlation Attention and State Space Modeling-based Approach for Precipitation Nowcasting[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250786

Citation:

ZHENG Hui, CHEN Fu, HE Shuping, QIU Xuexing, ZHU Hongfang, WANG Shaohua. A Multi-scale Spatiotemporal Correlation Attention and State Space Modeling-based Approach for Precipitation Nowcasting[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250786

Citation:

ZHENG Hui, CHEN Fu, HE Shuping, QIU Xuexing, ZHU Hongfang, WANG Shaohua. A Multi-scale Spatiotemporal Correlation Attention and State Space Modeling-based Approach for Precipitation Nowcasting[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250786

PDF( 8292 KB)

A Multi-scale Spatiotemporal Correlation Attention and State Space Modeling-based Approach for Precipitation Nowcasting

doi: 10.11999/JEIT250786 cstr: 32379.14.JEIT250786

1.
XXXXX
2.
XXXXX
3.
XXXXX
4.
XXXXX

Funds: National Natural Science Foundation of China (62476260, 62473003, 42471495), National Key Research and Development Program Project (2023YFF0805904), Science and Technology Innovation Plan Project of Anhui Province (202423l10050058)

Accepted Date: 2025-12-22
Rev Recd Date: 2025-12-22

Available Online: 2026-01-03

Abstract

Abstract

Objective Precipitation nowcasting, as one of the most representative tasks in the field of meteorological forecasting, uses radar echoes or precipitation sequences to predict precipitation distribution in the next 0-2 hours. It provides scientific and technological support for disaster warning and key decision-making, and maximizes the protection of people's lives and property. Current mainstream methods generally have problems such as loss of local details, inadequate representation of conditional information, and insufficient adaptability to complex areas. Therefore, this paper proposes a PredUMamba model based on the diffusion model. In this model, on the one hand, a Mamba block based on an adaptive zigzag scanning mechanism is introduced, which not only fully mines the key local detail information but also effectively reduces the computational complexity. On the other hand, a multi-scale spatio-temporal correlation attention model is designed to enhance the interaction ability of spatio-temporal hierarchical features while achieving a comprehensive representation of conditional information. More importantly, a radar echo dataset tailored for precipitation nowcasting in complex regions was constructed, specifically a radar dataset from the southern Anhui mountainous area, to validate the model's ability to accurately predict sudden, extreme rainfall events in complex areas. This research provides a new intelligent solution and theoretical support for precipitation nowcasting. Methods The PredUMamba model proposed in this paper adopts a two-stage diffusion model network. In the first stage, a frame-by-frame Variational Auto Encoder (VAE) is trained to map precipitation data in pixel space to a low-dimensional latent space. In the second stage, a diffusion network is constructed on the latent space after VAE encoding. In the diffusion network, this paper proposes an adaptive zigzag Mamba module, which adopts a spatio-temporal alternating adaptive zigzag scanning strategy, in which sequential scanning is performed within the rows of the data block and turn-back scanning is performed between rows, effectively capturing the detailed features of the precipitation field while maintaining low computational complexity. In addition, this paper designs a multi-scale spatio-temporal correlation attention module on both temporal and spatial scales. On the temporal scale, adaptive convolution kernels and convolution layers containing attention mechanisms are used to capture local and global information. On the spatial scale, a lightweight correlation attention is designed to aggregate spatial information, thus enhancing the ability to mine historical conditional information. Finally, this paper constructs a radar dataset for the southern Anhui mountainous area for the precipitation nowcasting task in complex terrain areas, which helps to verify the adaptability of the PredUMamba model and other models in the field to complex terrain areas. Results and Discussions In the PredUMamba model, by designing the adaptive zigzag Mamba module and the multi-scale spatio-temporal correlation attention module, the mining capability of the intrinsic spatio-temporal jointness of the data is enhanced, which can more accurately capture the characteristics of the conditional information and make prediction results that are more in line with the actual situation. Experimental results show that the PredUMamba model achieves the best performance in all indicators on the Southern Anhui Mountain Area and Shanghai radar datasets. On the SEVIR dataset, FVD, CSI_pool4, and CSI_pool16 are all superior to other methods, the CSI and CRPS also achieve very competitive results. In addition, further visualization prediction results show that PredUMamba's prediction results do not blur over time (Fig. 4), which indicates that the model has higher stability, and also has significant advantages in detail generation and overall motion trend capture, which indicates that the model can better generate edge details aligned with real precipitation conditions while maintaining accurate motion pattern predictions. Conclusions This paper proposes an innovative PredUMamba model based on a diffusion network architecture. The model significantly improves the model performance by introducing the Mamba module with adaptive zigzag scanning mechanism and the multi-scale spatio-temporal correlation attention module. The adaptive zigzag scanning Mamba module effectively captures the fine-grained spatio-temporal characteristics of precipitation data through a scanning strategy that alternates time and space, while reducing computational complexity. The multi-scale spatio-temporal correlation attention module enhances the ability to mine historical conditional information through a dual-branch network in the time dimension and a lightweight correlation attention mechanism in the spatial dimension, realizing the joint representation of local and global features. In order to verify the applicability of the model in complex terrain areas, this paper also constructed a radar dataset for the southern Anhui mountainous area. This dataset covers precipitation information under various terrain conditions and provides important support for extreme precipitation prediction in complex terrain areas. In addition, this study further conducts comparative experiments on the constructed dataset and some public datasets in the field. The experimental results show that the PredUMamba model achieved the best results in all indicators on the southern Anhui mountainous area and Shanghai radar datasets. On the SEVIR dataset, FVD, CSI_pool4 and CSI_pool16 all outperformed other methods, and the CRPS and CSI also achieved very competitive results. However, this study is only designed around a purely data-driven intelligent forecasting method, future work will focus on combining physical condition constraint information to improve the interpretability of the model and further optimize the prediction accuracy of small and medium-scale convective systems.
- Precipitation nowcasting,
- Diffusion model,
- State-space model,
- Radar echo,
- Multi-scale.

FullText(HTML)

References(30)

References

[1]	李海, 冯开泓, 杨文恒, 等. 机载双极化气象雷达多种降水粒子回波仿真方法研究[J]. 电子与信息学报, 2023, 45(8): 2945–2954. doi: 10.11999/JEIT220830. LI Hai, FENG Kaihong, YANG Wenheng, et al. Study on simulation method of precipitation particle echo of airborne dual-polarization weather radar[J]. Journal of Electronics & Information Technology, 2023, 45(8): 2945–2954. doi: 10.11999/JEIT220830.
[2]	周康辉, 郑永光, 杨波, 等. 强对流天气客观临近预报技术进展和展望[J]. 气象学报, 2025, 83(3): 798–812. doi: 10.11676/qxxb2025.20240106. ZHOU Kanghui, ZHENG Yongguang, YANG Bo, et al. Objective nowcasting of severe convective weather: Technological progress and outlook[J]. Acta Meteorologica Sinica, 2025, 83(3): 798–812. doi: 10.11676/qxxb2025.20240106.
[3]	朱平, 李生辰, 王振会, 等. 青藏高原东部暴雨云团局地强降水响应特征[J]. 遥感学报, 2014, 18(2): 405–431. doi: 10.11834/jrs.20143069. ZHU Ping, LI Shengchen, WANG Zhenhui, et al. Characteristics of rainstorm cloud clusters to local heavy precipitation over the eastern Qinghai-Tibet Plateau[J]. Journal of Remote Sensing, 2014, 18(2): 405–431. doi: 10.11834/jrs.20143069.
[4]	刘西川, 宋堃, 高太长, 等. 复杂大气条件对微波传播衰减的影响研究[J]. 电子与信息学报, 2018, 40(1): 181–188. doi: 10.11999/JEIT170253. LIU Xichuan, SONG Kun, GAO Taichang, et al. Research on the effect of complex atmospheric condition on microwave propagation attenuation[J]. Journal of Electronics & Information Technology, 2018, 40(1): 181–188. doi: 10.11999/JEIT170253.
[5]	BAILEY L P, CLARE M A, HUNT J E, et al. Highly variable deep-sea currents over tidal and seasonal timescales[J]. Nature Geoscience, 2024, 17(8): 787–794. doi: 10.1038/s41561-024-01494-2.
[6]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]. Proceedings of the 28th International Conference on Neural Information Processing Systems, Montréal, Canda, 2014: 2672–2680.
[7]	VAN DEN OORD A, KALCHBRENNER N, VINVALS O, et al. Conditional image generation with PixelCNN decoders[C]. Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Spain, 2016: 4797–4805.
[8]	GAO Zhihan, SHI Xingjian, HAN Boran, et al. PreDiff: Precipitation nowcasting with latent diffusion models[C]. Proceedings of the 37th International Conference on Neural Information Processing Systems, New Orleans, USA, 2023: 3439.
[9]	GONG Junchao, BAI Lei, YE Peng, et al. CasCast: Skillful high-resolution precipitation nowcasting via cascaded modelling[C]. Proceedings of the 41st International Conference on Machine Learning, 2024: 633. (查阅网上资料, 未找到本条文献出版地信息, 请确认).
[10]	LING Xudong, LI Chaorong, QIN Fengqing, et al. RNDiff: Rainfall nowcasting with condition diffusion model[J]. Pattern Recognition, 2025, 160: 111193. doi: 10.1016/j.patcog.2024.111193.
[11]	李云, 杨松林, 邢智童, 等. 多尺度特征注意力网络下的卫星信号识别研究[J]. 电子与信息学报, 2025, 47(6): 1792–1802. doi: 10.11999/JEIT250126. LI Yun, YANG Songlin, XING Zhitong, et al. Study on satellite signal recognition with multi-scale feature attention network[J]. Journal of Electronics & Information Technology, 2025, 47(6): 1792–1802. doi: 10.11999/JEIT250126.
[12]	ROMBACH R, BLATTMANN A, LORENZ D, et al. High-resolution image synthesis with latent diffusion models[C]. Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA 2022: 10684–10695. doi: 10.1109/CVPR52688.2022.01042.
[13]	CHEN Lei, CAO Yuan, MA Leiming, et al. A deep learning-based methodology for precipitation nowcasting with radar[J]. Earth and Space Science, 2020, 7(2): e2019EA000812. doi: 10.1029/2019EA000812.
[14]	VEILLETTE M S, SAMSI S, and MATTIOLI C J. SEVIR: A storm event imagery dataset for deep learning applications in radar and satellite meteorology[C]. Proceedings of the 34th International Conference on Neural Information Processing Systems, Vancouver, Canada, 2020: 1846.
[15]	PALMER T N, SHUTTS G J, HAGEDORN R, et al. Representing model uncertainty in weather and climate prediction[J]. Annual Review of Earth and Planetary Sciences, 2005, 33: 163–193. doi: 10.1146/annurev.earth.33.092203.122552.
[16]	GAO Zhihan, SHI Xingjian, WANG Hao, et al. Earthformer: Exploring space-time transformers for earth system forecasting[C]. Proceedings of the 36th Conference on Neural Information Processing Systems, New Orleans, USA, 2022: 25390–25403.
[17]	TANG Yujin, QI Lu, XIE Fei, et al. Video prediction transformers without recurrence or convolution[EB/OL]. https://arxiv.org/abs/2410.04733, 2024.
[18]	ZHANG Yuchen, LONG Mingsheng, CHEN Kaiyuan, et al. Skilful nowcasting of extreme precipitation with NowcastNet[J]. Nature, 2023, 619(7970): 526–532. doi: 10.1038/s41586-023-06184-4.
[19]	LEINONEN J, HAMANN U, NERINI D, et al. Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification[EB/OL]. https://arxiv.org/abs/2304.12891, 2023.
[20]	GAO Zhangyang, TAN Cheng, WU Lirong, et al. SimVP: Simpler yet better video prediction[C]. Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, USA, 2022: 3170–3180. doi: 10.1109/CVPR52688.2022.00317.
[21]	SHI Xingjian, GAO Zhihan, LAUSEN L, et al. Deep learning for precipitation nowcasting: A benchmark and a new model[C]. Proceedings of the 31st Conference on Neural Information Processing Systems, Long Beach, USA, 2017.
[22]	SHI Xingjian, CHEN Zhourong, WANG Hao, et al. Convolutional LSTM network: A machine learning approach for precipitation nowcasting[C]. Proceedings of the 29th International Conference on Neural Information Processing Systems, Montreal, Canada, 2015: 802–810.
[23]	ZHU Lianghui, LIAO Bencheng, ZHANG Qian, et al. Vision mamba: Efficient visual representation learning with bidirectional state space model[C]. Proceedings of the 41st International Conference on Machine Learning, Vienna, Austria, 2024: 2584.
[24]	CHEN Hongruixuan, SONG Jian, HAN Chengxi, et al. ChangeMamba: Remote sensing change detection with spatiotemporal state space model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 4409720. doi: 10.1109/TGRS.2024.3417253.
[25]	XU Xiongxiao, CHEN Canyu, LIANG Yueqing, et al. SST: Multi-scale hybrid mamba-transformer experts for time series forecasting[C]. Proceedings of the 34th ACM International Conference on Information and Knowledge Management, Seoul Republic of Korea, 2024: 3655–3665.
[26]	KINGMA D P, WELLING M. Auto-encoding variational bayes[C]. Proceedings of the 2nd International Conference on Learning Representations, Banff, Canada, 2014.
[27]	LARVOR G and BERTHOMIER L. MeteoNet: An open reference weather dataset for AI by Météo-France[C]. Proceedings of the 101st Annual AMS Meeting 2021, 2021. (查阅网上资料, 未找到对应的出版信息, 请确认).
[28]	LOSHCHILOV I and HUTTER F. Fixing weight decay regularization in Adam[EB/OL]. https://arxiv.org/abs/1711.05101, 2017.
[29]	WANG Yunbo, WU Haixu, ZHANG Jianjin, et al. PredRNN: A recurrent neural network for spatiotemporal predictive learning[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(2): 2208–2225. doi: 10.1109/TPAMI.2022.3165153.
[30]	YAN W, ZHANG Yunzhi, ABBEEL P, et al. VideoGPT: Video generation using VQ-VAE and transformers[EB/OL]. https://arxiv.org/abs/2104.10157, 2021.