高级搜索

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

融合空间自注意力感知的严重缺失多元时间序列插补算法

刘辉 冯浩然 马佳妮 郑红党 张林

刘辉, 冯浩然, 马佳妮, 郑红党, 张林. 融合空间自注意力感知的严重缺失多元时间序列插补算法[J]. 电子与信息学报. doi: 10.11999/JEIT250220
引用本文: 刘辉, 冯浩然, 马佳妮, 郑红党, 张林. 融合空间自注意力感知的严重缺失多元时间序列插补算法[J]. 电子与信息学报. doi: 10.11999/JEIT250220
LIU Hui, FENG Haoran, MA Jiani, ZHENG Hongdang, ZHANG Lin. Spatial Self-Attention Incorporated Imputation Algorithm for Severely Missing Multivariate Time Series[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250220
Citation: LIU Hui, FENG Haoran, MA Jiani, ZHENG Hongdang, ZHANG Lin. Spatial Self-Attention Incorporated Imputation Algorithm for Severely Missing Multivariate Time Series[J]. Journal of Electronics & Information Technology. doi: 10.11999/JEIT250220

融合空间自注意力感知的严重缺失多元时间序列插补算法

doi: 10.11999/JEIT250220 cstr: 32379.14.JEIT250220
基金项目: 国家自然科学基金(61971422),徐州市重点研发计划(社会发展)(KC22112)
详细信息
    作者简介:

    刘辉:男,副教授,博士,研究方向为大数据分析与处理、生物信息处理、无线通信

    冯浩然:男,硕士生,研究方向为大数据分析与处理、关联预测

    马佳妮:女,博士,研究方向为大数据分析与处理、生物信息处理

    郑红党:女,副教授,博士,研究方向为硬件系统设计、微波与天线

    张林:女,教授,博士,研究方向为大数据分析与处理、生物信息处理、多模态融合

    通讯作者:

    张林 lin.zhang@cumt.edu.cn

  • 中图分类号: TN92; TP181

Spatial Self-Attention Incorporated Imputation Algorithm for Severely Missing Multivariate Time Series

Funds: The National Natural Science Foundation of China (61971422), Xuzhou Science and Technology Innovation Plan - Key Special Project for Social Development (KC22112)
  • 摘要: 多元时间序列应用广泛,但极易发生缺失,影响相关规律的有效挖掘。已有插补方法大多面向低缺失率场景设计,应用至高缺失率场景通常面临梯度消失、时空依赖关系建模不足、复杂非线性特征表征困难等难题。该文提出一种融合空间自注意力感知的严重缺失多元时间序列插补算法(SSAImpute)。该算法采用双分支孪生结构,分别设计了空间自注意力感知和时域自注意力编码模块。其中,空间自注意力感知模块通过融合数据源位置等空间信息增强序列的相关性建模能力;时域自注意力编码模块设计了掩码自适应自注意力机制有效捕获时间层面的时间前后依赖性和特征相关性,避免了梯度消失现象。孪生分支之间通过动态加权融合,优化最终的插补输出。实现结果表明,与7个现有时间序列插补模型对比,该文所提方法在Inter-Sensor的4个子数据集均能有效提升严重缺失场景下的多元时间序列插补精度,在PeMS 3个子数据集的插补结果RMSE比次优方法分别提升4.1%, 6.7%和4.7%。该算法有望为严重缺失场景下的多元时间序列提供更准确的解决方案,进而为下游基于数据驱动的分析和决策任务提供更可靠的数据基础。
  • 图  1  SSAImpute的整体框架

    图  2  空间自注意力感知模块

    图  3  特征矩阵对角置零的空间动态自注意力机制

    图  4  时域自注意力编码模块

    图  5  掩码自适应自注意力机制

    图  6  模型在PeMS数据集不同缺失率上的插补性能

    图  7  3个数据集的时间序列插补可视化

    表  1  SSAImpute模型参数设置

    参数 PeMS04 PeMS07 PeMS11 Inter-Sensor
    输入序列长度 240 240 240 240
    自注意力头数 4 4 4 4
    堆叠层数 1 1 1 1
    隐藏层维度 512 512 256 128
    前馈网络隐藏单元数 128 32 32 512
    下载: 导出CSV

    表  2  SSAImpute在PeMS数据集上的消融实验结果

    方法 PeMS04 PeMS07 PeMS11
    MAE RMSE MRE(%) MAE RMSE MRE(%) MAE RMSE MRE(%)
    SSAImpute-SMD 0.206 0.334 22.7 0.161 0.275 18.4 0.181 0.285 19.7
    SSAImpute-S 0.208 0.333 22.9 0.159 0.282 18.1 0.180 0.284 19.7
    SSAImpute-TSAE 0.209 0.330 23.1 0.157 0.276 17.9 0.180 0.290 19.6
    SSAImpute-Pos 0.209 0.336 23.0 0.160 0.284 18.2 0.182 0.286 19.8
    SSAImpute-Fixed 0.207 0.332 22.8 0.159 0.283 18.2 0.181 0.288 19.8
    SSAImpute 0.203 0.328 22.4 0.153 0.274 17.5 0.177 0.282 19.3
    注:SSAImpute-SMD:去除SMD模块;SSAImpute-S:其中S代表Single,为单分支结构;SSAImpute-TSAE:去除时域自注意力编码模块;SSAImpute-Pos:去除位置信息;SSAImpute-Fixed:使用固定权重进行双分支融合。
    下载: 导出CSV

    表  3  模型在Inter-Sensor数据集上的消融实验结果

    方法TemperatureHumidityLightVoltage
    MAERMSEMRE(%)MAERMSEMRE(%)MAERMSEMRE(%)MAERMSEMRE(%)
    SSAImpute-SMD0.1180.51724.20.1130.48023.60.1410.26918.60.1300.82936.8
    SSAImpute-S0.1200.51524.60.1160.48624.10.1430.27118.90.1290.83036.6
    SSAImpute-TSAE0.1190.52024.50.1180.48324.60.1500.28519.80.1370.82939.0
    SSAImpute-Pos0.1210.51424.80.1150.48124.10.1450.27919.30.1290.83336.7
    SSAImpute-Fixed0.1170.51523.90.1170.48024.40.1420.27418.80.1320.82637.7
    SSAImpute0.1110.51322.60.1100.47722.90.1380.26515.30.1240.82735.2
    下载: 导出CSV

    表  4  模型在PeMS数据集上的插补性能

    方法 PeMS04 PeMS07 PeMS11
    MAE RMSE MRE(%) MAE RMSE MRE(%) MAE RMSE MRE(%)
    Mean[14] 0.893 1.014 98.5 0.861 0.992 98.1 0.906 1.040 98.7
    Median[14] 0.907 1.029 99.9 0.876 1.001 99.9 0.918 1.055 99.9
    KNN[36] 0.646 0.754 71.3 0.618 0.73 70.6 0.653 0.767 71.2
    M-RNN[25] 0.270 0.407 29.8 0.241 0.375 27.5 0.233 0.352 25.3
    BRITS[26] 0.212 0.347 23.4 0.170 0.301 19.4 0.192 0.303 20.9
    Transformer[37] 0.208 0.342 22.9 0.167 0.287 19.0 0.188 0.296 20.5
    SAITS[30] 0.209 0.348 23.0 0.164 0.296 18.7 0.186 0.299 20.2
    SSAImpute 0.203 0.328 22.4 0.153 0.274 17.5 0.177 0.282 19.3
    下载: 导出CSV

    表  5  模型在Inter-Sensor数据集上的插补性能

    方法TemperatureHumidityLightVoltage
    MAERMSEMRE
    (%)
    MAERMSEMRE
    (%)
    MAERMSEMRE
    (%)
    MAERMSEMRE
    (%)
    Mean[14]0.4750.73997.00.4630.69896.70.7610.9281000.2450.82969.7
    Median[14]0.4890.772100.00.4780.73399.90.7550.93999.90.2010.82969.7
    KNN[36]0.3020.62461.60.2910.58160.70.3750.57249.60.2220.82363.2
    M-RNN[25]0.3000.58261.20.2900.53960.60.3050.46040.40.3170.83689.9
    BRITS[26]0.1940.53839.80.1870.49939.00.1840.32924.40.1360.82438.7
    Transformer[37]0.1910.52439.00.1860.49439.00.1630.30421.60.1430.82440.7
    SAITS[30]0.1320.51427.10.1260.48226.40.1530.29520.20.1360.83038.7
    SSAImpute0.1110.51322.60.1120.47722.90.1380.26515.30.1240.82735.2
    下载: 导出CSV
  • [1] LUO Yonghong, CAI Xiangrui, ZHANG Ying, et al. Multivariate time series imputation with generative adversarial networks[C]. The 32nd International Conference on Neural Information Processing Systems, Montréal, Canada, 2018: 1603–1614.
    [2] UCHIHARA M, TANABE A, and KAJIO H. Clinical issues and suggestions: Dashboard visualization of the trajectory of patients with malignant hormone-producing tumors for precision medicine[C]. 2023 Workshop on Visual Analytics in Healthcare (VAHC), Melbourne, Australia, 2023: 47–49. doi: 10.1109/VAHC60858.2023.00015.
    [3] PAL R, ADHIKARI D, HEYAT M B B, et al. Yoga meets intelligent internet of things: Recent challenges and future directions[J]. Bioengineering, 2023, 10(4): 459. doi: 10.3390/bioengineering10040459.
    [4] 骆阳, 张旗. 基于模糊关联规则的海量气象数据动态挖掘[J]. 电子设计工程, 2023, 31(22): 149–152. doi: 10.14022/j.issn1674-6236.2023.22.031.

    LUO Yang and ZHANG Qi. Dynamic mining of massive meteorological data based on fuzzy association rules[J]. Electronic Design Engineering, 2023, 31(22): 149–152. doi: 10.14022/j.issn1674-6236.2023.22.031.
    [5] WANG Yunsheng, XU Xinghan, HU Lei, et al. A time series continuous missing values imputation method based on generative adversarial networks[J]. Knowledge-Based Systems, 2024, 283: 111215. doi: 10.1016/j.knosys.2023.111215.
    [6] XU Longfei, XU Lingyu, and YU Jie. A multi-task learning-based generative adversarial network for red tide multivariate time series imputation[J]. Complex & Intelligent Systems, 2023, 9(2): 1363–1376. doi: 10.1007/s40747-022-00856-w.
    [7] MIAO Xiaoye, WU Yangyang, CHEN Lu, et al. An experimental survey of missing data imputation algorithms[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 35(7): 6630–6650. doi: 10.1109/TKDE.2022.3186498.
    [8] LIN Weichao, TSAI C F, and ZHONG Jiarong. Deep learning for missing value imputation of continuous data and the effect of data discretization[J]. Knowledge-Based Systems, 2022, 239: 108079. doi: 10.1016/j.knosys.2021.108079.
    [9] 郭艳, 宋晓祥, 李宁, 等. 多变量时间序列中基于克罗内克压缩感知的缺失数据预测算法[J]. 电子与信息学报, 2019, 41(4): 858–864. doi: 10.11999/JEIT180541.

    GUO Yan, SONG Xiaoxiang, LI Ning, et al. Missing data prediction based on Kronecker compressing sensing in multivariable time series[J]. Journal of Electronics & Information Technology, 2019, 41(4): 858–864. doi: 10.11999/JEIT180541.
    [10] LIU Hui, YU Jian, CHEN Xiangzhi, et al. NeuMF: Predicting anti-cancer drug response through a neural matrix factorization model[J]. Current Bioinformatics, 2022, 17(9): 835–847. doi: 10.2174/1574893617666220609114052.
    [11] 朱宇航, 刘树新, 吉立新, 等. 一种融合局部拓扑影响力的时序链路预测算法[J]. 电子与信息学报, 2022, 44(4): 1440–1452. doi: 10.11999/JEIT210019.

    ZHU Yuhang, LIU Shuxin, JI Lixin, et al. A temporal link predict algorithm based on fusion local structure influence[J]. Journal of Electronics & Information Technology, 2022, 44(4): 1440–1452. doi: 10.11999/JEIT210019.
    [12] XU Meng, DI Yining, DING Hongxing, et al. AGNP: Network-wide short-term probabilistic traffic speed prediction and imputation[J]. Communications in Transportation Research, 2023, 3: 100099. doi: 10.1016/j.commtr.2023.100099.
    [13] LIU Hui, WANG Feng, YU Jian, et al. DBDNMF: A dual branch deep neural matrix factorization method for drug response prediction[J]. PLoS Computational Biology, 2024, 20(4): e1012012. doi: 10.1371/journal.pcbi.1012012.
    [14] SAMAD M D, ABRAR S, and DIAWARA N. Missing value estimation using clustering and deep learning within multiple imputation framework[J]. Knowledge-Based Systems, 2022, 249: 108968. doi: 10.1016/j.knosys.2022.108968.
    [15] ZHANG Weibin, ZHANG Pulin, YU Yinghao, et al. Missing data repairs for traffic flow with self-attention generative adversarial imputation net[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(7): 7919–7930. doi: 10.1109/TITS.2021.3074564.
    [16] THOMAS T and RAJABI E. A systematic review of machine learning-based missing value imputation techniques[J]. Data Technologies and Applications, 2021, 55(4): 558–585. doi: 10.1108/DTA-12-2020-0298.
    [17] XUE Yu, TANG Yihang, XU Xin, et al. Multi-objective feature selection with missing data in classification[J]. IEEE Transactions on Emerging Topics in Computational Intelligence, 2022, 6(2): 355–364. doi: 10.1109/TETCI.2021.3074147.
    [18] KIM H, GOLUB G H, and PARK H. Missing value estimation for DNA microarray gene expression data: Local least squares imputation[J]. Bioinformatics, 2005, 21(2): 187–198. doi: 10.1093/bioinformatics/bth499.
    [19] CAMASTRA F, CAPONE V, CIARAMELLA A, et al. Prediction of environmental missing data time series by support vector machine regression and correlation dimension estimation[J]. Environmental Modelling & Software, 2022, 150: 105343. doi: 10.1016/j.envsoft.2022.105343.
    [20] LIN Weichao and TSAI C F. Missing value imputation: A review and analysis of the literature (2006–2017)[J]. Artificial Intelligence Review, 2020, 53(2): 1487–1509. doi: 10.1007/s10462-019-09709-4.
    [21] DABERDAKU S, TAVAZZI E, and DI CAMILLO B. A combined interpolation and weighted K-Nearest Neighbours approach for the imputation of longitudinal ICU laboratory data[J]. Journal of Healthcare Informatics Research, 2020, 4(2): 174–188. doi: 10.1007/s41666-020-00069-1.
    [22] FANG Le, XIANG Wei, ZHOU Yuan, et al. Dual-branch cross-dimensional self-attention-based imputation model for multivariate time series[J]. Knowledge-Based Systems, 2023, 279: 110896. doi: 10.1016/j.knosys.2023.110896.
    [23] QIN Rui and WANG Yong. ImputeGAN: Generative adversarial network for multivariate time series imputation[J]. Entropy, 2023, 25(1): 137. doi: 10.3390/e25010137.
    [24] CHE Zhengping, PURUSHOTHAM S, CHO K, et al. Recurrent neural networks for multivariate time series with missing values[J]. Scientific Reports, 2018, 8(1): 6085. doi: 10.1038/s41598-018-24271-9.
    [25] YOON J, ZAME W R, and VAN DER SCHAAR M. Estimating missing data in temporal data streams using multi-directional recurrent neural networks[J]. IEEE Transactions on Biomedical Engineering, 2019, 66(5): 1477–1490. doi: 10.1109/TBME.2018.2874712.
    [26] CAO Wei, WANG Dong, LI Jian, et al. BRITS: Bidirectional recurrent imputation for time series[C]. The 32nd International Conference on Neural Information Processing Systems, Montréal, Canada, 2018: 6776–6786.
    [27] SUO Qiuling, ZHONG Weida, XUN Guangxu, et al. GLIMA: Global and local time series imputation with multi-directional attention learning[C]. 2020 IEEE International Conference on Big Data (Big Data), Atlanta, USA, 2020: 798–807. doi: 10.1109/BigData50022.2020.9378408.
    [28] MA Jiawei, SHOU Zheng, ZAREIAN A, et al. CDSA: Cross-dimensional self-attention for multivariate, geo-tagged time series imputation[EB/OL]. https://arxiv.org/abs/1905.09904, 2019.
    [29] SHAN Siyuan, LI Yang, and OLIVA J B. NRTSI: Non-recurrent time series imputation[C]. ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Rhodes Island, Greece, 2023: 1–5. doi: 10.1109/ICASSP49357.2023.10095054.
    [30] DU Wenjie, CÔTÉ D, and LIU Yan. SAITS: Self-attention-based imputation for time series[J]. Expert Systems with Applications, 2023, 219: 119619. doi: 10.1016/j.eswa.2023.119619.
    [31] CHUI C K, MHASKAR H N, and VAN DER WALT M D. Data-driven atomic decomposition via frequency extraction of intrinsic mode functions[J]. GEM - International Journal on Geomathematics, 2016, 7(1): 117–146. doi: 10.1007/s13137-015-0079-3.
    [32] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]. The 31st International Conference on Neural Information Processing Systems, Long Beach, USA, 2017: 6000–6010.
    [33] PAN Zhuofu, WANG Yalin, WANG Kai, et al. Intel lab data[EB/OL]. https://db.csail.mit.edu/labdata/labdata.html, 2023.
    [34] 潘立强, 李建中, 骆吉洲. 传感器网络中一种基于时-空相关性的缺失值估计算法[J]. 计算机学报, 2010, 33(1): 1–11. doi: 10.3724/SP.J.1016.2010.00001.

    PAN Liqiang, LI Jianzhong, and LUO Jizhou. A temporal and spatial correlation based missing values imputation algorithm in wireless sensor networks[J]. Chinese Journal of Computers, 2010, 33(1): 1–11. doi: 10.3724/SP.J.1016.2010.00001.
    [35] BOX G E P, JENKINS G M, REINSEL G C, et al. Time Series Analysis: Forecasting and Control[M]. 5th ed. Hoboken: Wiley, 2015.
    [36] ZHANG Shichao. Nearest neighbor selection for iteratively kNN imputation[J]. Journal of Systems and Software, 2012, 85(11): 2541–2552. doi: 10.1016/j.jss.2012.05.073.
    [37] NIE Tong, QIN Guoyang, MA Wei, et al. ImputeFormer: Low rankness-induced transformers for generalizable spatiotemporal imputation[C]. The 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Barcelona, Spain, 2024: 2260–2271. doi: 10.1145/3637528.3671751.
  • 加载中
图(7) / 表(5)
计量
  • 文章访问数:  87
  • HTML全文浏览量:  30
  • PDF下载量:  23
  • 被引次数: 0
出版历程
  • 收稿日期:  2025-03-31
  • 修回日期:  2025-09-09
  • 网络出版日期:  2025-09-16

目录

    /

    返回文章
    返回