2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于不确定性的多元时间序列分类算法研究

张旭 张亮 金博 张红哲

张旭, 张亮, 金博, 张红哲. 基于不确定性的多元时间序列分类算法研究. 自动化学报, 2023, 49(4): 790−804 doi: 10.16383/j.aas.c210302
引用本文: 张旭, 张亮, 金博, 张红哲. 基于不确定性的多元时间序列分类算法研究. 自动化学报, 2023, 49(4): 790−804 doi: 10.16383/j.aas.c210302
Zhang Xu, Zhang Liang, Jin Bo, Zhang Hong-Zhe. Uncertainty-based multivariate time series classification. Acta Automatica Sinica, 2023, 49(4): 790−804 doi: 10.16383/j.aas.c210302
Citation: Zhang Xu, Zhang Liang, Jin Bo, Zhang Hong-Zhe. Uncertainty-based multivariate time series classification. Acta Automatica Sinica, 2023, 49(4): 790−804 doi: 10.16383/j.aas.c210302

基于不确定性的多元时间序列分类算法研究

doi: 10.16383/j.aas.c210302
基金项目: 国家自然科学基金 (61772110), 辽宁省教育厅科学研究经费 (LJKZ1045), 上海市卫生和计划生育委员会科研课题(20184Y0247)资助
详细信息
    作者简介:

    张旭:大连理工大学机械工程学院硕士研究生. 主要研究方向为机器学习, 数据挖掘与应用. E-mail: zhangxu1@mail.dlut.edu.cn

    张亮:东北财经大学国际商学院讲师. 主要研究方向为多元时间序列挖掘, 医疗健康大数据. 本文通信作者. E-mail: liang.zhang@dufe.edu.cn

    金博:大连理工大学创新创业学院教授. 主要研究方向为信息检索, 数据挖掘和智能计算. E-mail: jinbo@dlut.edu.cn

    张红哲:大连理工大学机械工程学院副教授. 主要研究方向为工业大数据的挖掘与应用. E-mail: zhanghongzhe@dlut.edu.cn

Uncertainty-based Multivariate Time Series Classification

Funds: Supported by National Natural Science Foundation of China (61772110), Scientific Research Project of the Education Department of Liaoning Province (LJKZ1045), and Scientific Research Project of Shanghai Health and Family Planning Commission (20184Y0247)
More Information
    Author Bio:

    ZHANG Xu Master student at the Mechanical Engineering College, Dalian University of Technology. His research interest covers machine learning, data mining and applications

    ZHANG Liang Lecturer at the International Business College, Dongbei University of Finance and Economics. His research interest covers multivariate time series mining and healthcare big data analytics. Corresponding author of this paper

    JIN Bo Professor at the School of Innovation and Entrepreneurship, Dalian University of Technology. His research interest covers information retrieval, data mining, and intelligent computing

    ZHANG Hong-Zhe Associate professor at the Mechanical Engineering College, Dalian University of Technology. Her research interest covers industrial big data mining and application

  • 摘要: 多元时间序列(Multivariate time series, MTS)分类是许多领域中的重要问题, 准确的分类结果可以有效地帮助决策. 当前的MTS分类算法在个体的表征学习阶段难以自动建模多元变量之间复杂的交互关系, 并且无法评估分类结果的可信度, 这会导致模型性能受限, 以及缺乏具备统计意义的可靠性解释. 本文提出了一种基于不确定性的多元时间序列分类算法, 变分贝叶斯共享图神经网络, 即VBSGNN (Variational Bayes shared graph neural network). 首先通过图神经网络(Graph neural network, GNN)提取多元变量之间的交互特征, 然后利用贝叶斯神经网络(Bayesian neural network, BNN)为预测过程引入了不确定性. 最后在10个公开MTS数据集上进行了算法实验, 并与当前提出的7类算法进行了比较, 结果表明VBSGNN可有效学习多元变量之间的交互关系, 提升了分类效果, 并使得模型具备一定的可靠性评估能力.
    1)  1 数据获取地址: http://timeseriesclassification.com
  • 图  1  MTS数据特征提取模型架构与优化流程

    Fig.  1  Feature extraction model architecture and optimization process of MTS data

    图  2  随机变分推断流程

    Fig.  2  The process of stochastic variational inference

    图  3  SGNN-T和VBSGNN的预测分布方差对比

    Fig.  3  Variance comparison of prediction distribution between SGNN-T and VBSGNN

    图  4  基于VBSGNN不确定得分改善预测效果评估

    Fig.  4  Evaluation of improving prediction effect based on VBSGNN uncertainty score

    图  5  基于VBSGNN的模型不确定性估计 (NATO数据集)

    Fig.  5  Model uncertainty estimation based on the VBSGNN (NATO dataset)

    图  6  节点大小与边连接的关系 (NATO数据集)

    Fig.  6  The relationship between node size and edge connection (NATO dataset)

    图  7  神经网络学习到的单类别与多类别共享图结构 (NATO数据集)

    Fig.  7  Single class and multi class shared graph structures learned by neural networks (NATO dataset)

    表  1  实验中使用的10个数据集概要

    Table  1  Summary of the 10 UEA datasets used in experimentation

    名称训练集大小测试集大小多变量维度时间维度类别个数
    AFAtrialFibrillation151526403
    FMFingerMovements31610028502
    HMDHandMovementDirection16074104004
    HBHeartbeat204205614052
    LIBLibras18018024515
    MIMotorImagery2781006430002
    NATONATOPS18018024516
    PDPenDigits749434982810
    SRS2SelfRegulationSCP2200180711522
    SWJStandWalkJump1215425003
    下载: 导出CSV

    表  2  在10个公开数据集上的不同算法准确率对比

    Table  2  Accuracy of different algorithms on 10 public datasets are compared

    算法数据集
    AFFMHMDHBLIBMINATOPDSRS2SWJWins
    ED0.2670.5190.2790.6200.8330.5100.8500.9730.4830.3330
    DTWI0.2670.5130.2970.6590.8940.3900.8500.9390.5330.2000
    DTWD0.2670.5290.2310.7170.8720.5000.8830.9770.5390.2000
    ED(norm)0.2000.5100.2780.6190.8330.5100.8500.9730.4830.3330
    DTWI(norm)0.2670.5200.2970.6580.8940.3900.8500.9390.5330.2000
    DTWD(norm)0.2670.5300.2310.7170.8700.5000.8830.9770.5390.2000
    WEASEL+MUSE0.4000.5500.3650.7270.8940.5000.8700.9480.4600.2670
    HIVE-COTE0.1330.5500.4460.7220.9000.6100.8890.9340.4610.3331
    MLSTM-FCN0.3330.5800.5270.6630.8500.5100.9000.9780.4720.4000
    TapNet0.3330.4700.3380.7510.8780.5900.9390.9800.5500.1330
    MTPool-M0.5330.5040.4860.7420.8280.5600.9280.9780.5500.5330
    MTPool-D0.4000.5300.4590.7370.8110.6000.9440.9770.5500.5330
    MTPool-S0.4000.5900.4730.7220.8110.5400.8890.9830.5390.6670
    MTPool-One0.4000.5700.4050.7170.8330.5400.8890.9700.5390.6000
    MTPool-Corr0.4000.5900.4190.7220.8280.5600.9040.9730.5500.6000
    MTPool0.4670.6200.4320.7420.8610.6300.9040.9830.6000.6670
    SGNN-S0.6000.6500.5410.7410.8890.6000.9610.9840.5890.6002
    SGNN-I0.5330.5500.5140.7410.8830.6400.9330.9740.5720.6001
    SGNN-A0.5330.5600.5000.7510.8780.5600.9610.9800.5500.6000
    SGNN-T0.6000.6400.6080.7560.8890.6300.9780.9850.6000.7337
    VBSGNN0.6670.6800.6220.7760.8720.6800.9720.9840.6220.7339
    下载: 导出CSV

    表  3  NATO图结构中24个节点对应的变量名称

    Table  3  Corresponding variable names of 24 nodes in graph structure based on NATO dataset

    手部传感器变量肘部传感器变量手腕传感器变量拇指传感器变量
    节点 0: 左手尖 X 坐标节点 6: 左肘部 X 坐标节点 12: 左手腕 X 坐标节点 18: 左拇指 X 坐标
    节点 1: 左手尖 Y 坐标节点 7: 左肘部 Y 坐标节点 13: 左手腕 Y 坐标节点 19: 左拇指 Y 坐标
    节点 2: 左手尖 Z 坐标节点 8: 左肘部 Z 坐标节点 14: 左手腕 Z 坐标节点 20: 左拇指 Z 坐标
    节点 3: 右手尖 X 坐标节点 9: 右肘部 X 坐标节点 15: 右手腕 X 坐标节点 21: 右拇指 X 坐标
    节点 4: 右手尖 Y 坐标节点 10: 右肘部 Y 坐标节点 16: 右手腕 Y 坐标节点 22: 右拇指 Y 坐标
    节点 5: 右手尖 Z 坐标节点 11: 右肘部 Z 坐标节点 17: 右手腕 Z 坐标节点 23: 右拇指 Z 坐标
    下载: 导出CSV
  • [1] 张熙来, 赵俭辉, 蔡波. 针对PM_2.5单时间序列数据的动态调整预测模型. 自动化学报, 2018, 44(10): 1790-1798

    Zhang Xi-Lai, Zhao Jian-Hui, Cai Bo. Prediction model with dynamic adjustment for single time series of PM_2.5. Acta Automatica Sinica, 2018, 44(10): 1790-1798
    [2] 徐任超, 阎威武, 王国良, 杨健程, 张曦. 基于周期性建模的时间序列预测方法及电价预测研究. 自动化学报, 2020, 46(6): 1136-1144

    Xu Ren-Chao, Yan Wei-Wu, Wang Guo-Liang, Yang Jian-Cheng, Zhang Xi. Time series forecasting based on seasonality modeling and its application to electricity price forecasting. Acta Automatica Sinica, 2020, 46(6): 1136-1144
    [3] Keogh E, Chu S, Hart D, Pazzani M. Segmenting time series: A survey and novel approach. Data Mining in Time Series Databases. Singapore: World Scientific, 2004. 1−21
    [4] Zhang X C, Gao Y F, Lin J, Lu C T. TapNet: Multivariate time series classification with attentional prototypical network. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, USA: AAAI, 2020. 6845−6852
    [5] Wang X, Gao Y F, Lin J, Rangwala H, Mittu R. A machine learning approach to false alarm detection for critical arrhythmia alarms. In: Proceedings of the IEEE 14th International Conference on Machine Learning and Applications (ICMLA). Miami, USA: IEEE, 2015. 202−207
    [6] Minnen D, Starner T, Essa I, Isbell C. Discovering characteristic actions from on-body sensor data. In: Proceedings of the 10th IEEE International Symposium on Wearable Computers. Montreux, Switzerland: IEEE, 2006. 11−18
    [7] Rakthanmanon T, Keogh E. Fast shapelets: A scalable algorithm for discovering time series shapelets. In: Proceedings of the 2013 SIAM International Conference on Data Mining. Austin, USA: SIAM, 2013. 668−676
    [8] Seto S, Zhang W Y, Zhou Y C. Multivariate time series classification using dynamic time warping template selection for human activity recognition. In: Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence. Cape Town, South Africa: IEEE, 2015. 1399−1406
    [9] Pei W J, Dibeklioğlu H, Tax D M J, Van Der Maaten L. Multivariate time-series classification using the hidden-unit logistic model. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(4): 920-931 doi: 10.1109/TNNLS.2017.2651018
    [10] Karim F, Majumdar S, Darabi H, Harford S. Multivariate LSTM-FCNs for time series classification. Neural Networks, 2019, 116: 237-245 doi: 10.1016/j.neunet.2019.04.014
    [11] Zheng Y, Liu Q, Chen E H, Ge Y, Zhao J L. Time series classification using multi-channels deep convolutional neural networks. In: Proceedings of the 15th International Conference on Web-Age Information Management. Macau, China: Springer, 2014. 298−310
    [12] 毛文涛, 蒋梦雪, 李源, 张仕光. 基于异常序列剔除的多变量时间序列结构化预测. 自动化学报, 2018, 44(4): 619-634

    Mao Wen-Tao, Jiang Meng-Xue, Li Yuan, Zhang Shi-Guang. Structural prediction of multivariate time series through outlier elimination. Acta Automatica Sinica, 2018, 44(4): 619-634
    [13] Wu Z H, Pan S R, Chen F W, Long G D, Zhang C Q, Yu P S. A comprehensive survey on graph neural networks. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1): 4-24 doi: 10.1109/TNNLS.2020.2978386
    [14] Scarselli F, Gori M, Tsoi A C, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80 doi: 10.1109/TNN.2008.2005605
    [15] Cao D F, Wang Y J, Duan J Y, Zhang C, Zhu X, Huang C R, et al. Spectral temporal graph neural network for multivariate time-series forecasting. In: Proceedings of the 34th Advances in Neural Information Processing Systems. arXiv: 2103.07719
    [16] Wu Z H, Pan S R, Long G D, Jiang J, Chang X J, Zhang C Q. Connecting the dots: Multivariate time series forecasting with graph neural networks. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event: ACM, 2020. 753−763
    [17] Duan Z H, Xu H Y, Wang Y Y, Huang Y D, Ren A N, Xu Z B, et al. Multivariate time-series classification with hierarchical variational graph pooling. arXiv preprint arXiv: 2010.05649, 2020.
    [18] Spadon G, Hong S D, Brandoli B, Matwin S, Rodrigues-Jr J F, Sun J M. Pay attention to evolution: Time series forecasting with deep graph-evolution learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, doi: 10.1109/TPAMI.2021.3076155
    [19] Hamilton W L, Ying R, Leskovec J. Inductive representation learning on large graphs. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: Curran Associates Inc., 2017. 1025−1035
    [20] Xu K, Hu W H, Leskovec J, Jegelka S. How powerful are graph neural networks. In: Proceedings of the 7th International Conference on Learning Representations. New Orleans, USA: ICLR, 2019.
    [21] Veličković P, Cucurull C, Casanova A, Romero A, Liò P, Bengio Y. Graph attention networks. In: Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada: ICLR, 2018.
    [22] Tran D, Dusenberry M W, Van Der Wilk M, Hafner D. Bayesian layers: A module for neural network uncertainty. In: Proceedings of the 33rd International Conference on Neural Information Processing Systems. Vancouver, Canada: Curran Associates Inc., 2019. 14660−14672
    [23] Kendall A, Gal Y. What uncertainties do we need in Bayesian deep learning for computer vision. In: Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, USA: 2017. 5575−5585
    [24] Zhang C, Bütepage J, Kjellström H, Mandt S. Advances in variational inference. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8): 2008-2026 doi: 10.1109/TPAMI.2018.2889774
    [25] Zhang X L, Qian B Y, Cao S L, Li Y, Chen H, Zheng Y F. INPREM: An interpretable and trustworthy predictive model for healthcare. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. Virtual Event: ACM, 2020. 450−460
    [26] Ranjan E, Sanyal S, Talukdar P P. ASAP: Adaptive structure aware pooling for learning hierarchical graph representations. In: Proceedings of the 34th AAAI Conference on Artificial Intelligence. New York, USA: AAAI, 2020. 5470−5477
    [27] Du J, Zhang S H, Wu G H, Moura J M F, Kar S. Topology adaptive graph convolutional networks. arXiv: 1710.10370, 2018.
    [28] Szegedy C, Liu W, Jia Y Q, Sermanet P, Reed S, Anguelov D, et al. Going deeper with convolutions. In: Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, USA: IEEE, 2015. 1−9
    [29] Hinton G E, Srivastava N, Krizhevsky A, et al. Improving neural networks by preventing co-adaptation of feature detectors. Computer Science, 2012, 3(4): 212-223
    [30] Kingma D P, Ba J. Adam: A method for stochastic optimization. In: Proceedings of the 3rd International Conference on Learning Representations. San Diego, USA: ICLR, 2015. 1−15
    [31] 季文强. 基于深度学习和不确定性量化的数据驱动剩余寿命预测方法研究 [硕士学位论文], 中国科学技术大学, 中国, 2020

    Ji Wen-Qiang. Research on Data-driven Remaining Useful Life Prediction Method Based on Deep Learning and Uncertainty Quantification [Master thesis], University of Science and Technology of China, China, 2020
    [32] Hoffman M D, Blei D M, Wang C, Paisley J W. Stochastic variational inference. Journal of Machine Learning Research, 2013, 14(1): 1303-1347
    [33] Zhang A Y, Zhou H H. Theoretical and computational guarantees of mean field variational inference for community detection. The Annals of Statistics, 2020, 48(5): 2575-2598
    [34] Shokoohi-Yekta M, Wang J, Keogh E J. On the non-trivial generalization of dynamic time warping to the multi-dimensional case. In: Proceedings of the 2015 SIAM International Conference on Data Mining. Vancouver, Canada: SIAM, 2015. 289−297
    [35] Bagnall A , Flynn M , Large J. A tale of two toolkits, report the third: On the usage and performance of HIVE-COTE v1.0. arXiv preprint arXiv: 2004.06069, 2020.
  • 加载中
图(7) / 表(3)
计量
  • 文章访问数:  1264
  • HTML全文浏览量:  1156
  • PDF下载量:  348
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-04-11
  • 录用日期:  2021-11-02
  • 网络出版日期:  2021-12-08
  • 刊出日期:  2023-04-20

目录

    /

    返回文章
    返回