2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

一种空间几何角度最大化的随机增量学习模型及应用

南静 代伟 袁冠 周平

南静, 代伟, 袁冠, 周平. 一种空间几何角度最大化的随机增量学习模型及应用. 自动化学报, 2023, 49(6): 1283−1294 doi: 10.16383/j.aas.c211041
引用本文: 南静, 代伟, 袁冠, 周平. 一种空间几何角度最大化的随机增量学习模型及应用. 自动化学报, 2023, 49(6): 1283−1294 doi: 10.16383/j.aas.c211041
Nan Jing, Dai Wei, Yuan Guan, Zhou Ping. A stochastic incremental learning model with maximizing spatial geometry angle and its application. Acta Automatica Sinica, 2023, 49(6): 1283−1294 doi: 10.16383/j.aas.c211041
Citation: Nan Jing, Dai Wei, Yuan Guan, Zhou Ping. A stochastic incremental learning model with maximizing spatial geometry angle and its application. Acta Automatica Sinica, 2023, 49(6): 1283−1294 doi: 10.16383/j.aas.c211041

一种空间几何角度最大化的随机增量学习模型及应用

doi: 10.16383/j.aas.c211041
基金项目: 国家自然科学基金(61973306), 江苏省自然科学基金优秀青年项目(BK20200086), 江苏省研究生科研与实践创新计划(KYCX22_2552), 中国矿业大学未来杰出人才助力计划(2022WLKXJ077)资助
详细信息
    作者简介:

    南静:中国矿业大学信息与控制工程学院博士研究生. 2021 年获得中国矿业大学硕士学位. 主要研究方向为随机权神经网络, 模式识别. E-mail: jingn@cumt.edu.cn

    代伟:中国矿业大学信息与控制工程学院教授. 2015 年获得东北大学博士学位. 主要研究方向为复杂工业过程建模、运行优化与控制. 本文通信作者. E-mail: weidai@cumt.edu.cn

    袁冠:中国矿业大学计算机科学与技术学院教授. 2012 年获得中国矿业大学博士学位. 主要研究方向为数据挖掘, 软件工程. E-mail: yuanguan@cumt.edu.cn

    周平:东北大学教授. 分别于2003年、2006年、2013年获得东北大学学士学位、硕士学位和博士学位. 主要研究方向为工业过程运行反馈控制, 数据驱动建模与控制. E-mail: zhouping@mail.neu.edu.cn

A Stochastic Incremental Learning Model With Maximizing Spatial Geometry Angle and Its Application

Funds: Supported by National Natural Science Foundation of China (61973306), Outstanding Youth Project of Jiangsu Provincial Natural Science Foundation (BK20200086), Postgraduate Research & Practice Innovation Program of Jiangsu Province (KYCX22_2552), and Assistance Program for Future Outstanding Talents of China University of Mining and Technology (2022WLKXJ077)
More Information
    Author Bio:

    NAN Jing Ph.D. candidate at the School of Information and Control Engineering, China University of Mining and Technology. He received his master degree from China University of Mining and Technology in 2021. His research interest covers random weight neural networks and pattern recognition

    DAI Wei Professor at the School of Information and Control Engineering, China University of Mining and Technology. He received his Ph.D. degree from Northeastern University in 2015. His research interest covers modeling, operational optimization, and control for complex industrial process. Corresponding author of this paper

    YUAN Guan Professor at the School of Computer Science & Technology, China University of Mining and Technology. He received his Ph.D. degree from China University of Mining and Technology in 2012. His research interest covers data mining and software engineering

    ZHOU Ping Professor at Northeastern University. He received his bachelor degree, master degree and Ph.D. degree from Northeastern University in 2003, 2006 and 2013, respectively. His research interest covers operation feedback control of industrial process and data-driven modeling and control

  • 摘要: 针对随机权神经网络(Random weight neural networks, RWNNs)隐含层节点随机生成过程可解释性不足和节点随机生成而导致的网络结构不紧致等问题, 提出了一种空间几何角度最大化随机增量学习模型(Stochastic incremental learning model with maximizing spatial geometry angle, SGA-SIM). 首先, 以空间几何视角深入分析随机增量学习过程, 建立了具有可解释性的空间几何角度最大化约束, 以改善隐含层节点质量, 并证明该学习模型具有无限逼近特性; 同时, 引入格雷维尔迭代法优化学习模型输出权值计算方法, 提高模型学习效率. 在真实的分类和回归数据集以及数值模拟实例上的实验结果表明, 所提增量学习模型在建模速度、模型精度和模型网络结构等多个方面具有明显优势.
  • 图  1  ${e_{L{\rm{ - }}1}}$, ${e_L}$和${g_L}$关系示意图

    Fig.  1  Relationship diagram of ${e_{L{\rm{ - }}1}}$, ${e_L}$and${g_L}$

    图  2  4种模型的收敛曲线图

    Fig.  2  Convergence curves of four models

    图  3  4种模型的拟合曲线图

    Fig.  3  Fitting curves of four models

    图  4  节点对拟合性能的影响

    Fig.  4  Effect of nodes on fitting performance

    图  5  手势识别系统框架图

    Fig.  5  Frame diagram of gesture recognition system

    图  6  智能手套框架图

    Fig.  6  Frame diagram of smart gloves

    图  7  软件部分结构图

    Fig.  7  Software structure diagram

    图  8  IRWNNs的三种指标结果对比图

    Fig.  8  Comparison chart of three index results of IRWNNs

    图  9  SGA-SIM-I的三种指标结果对比图

    Fig.  9  Comparison chart of three index results of SGA-SIM-I

    图  10  SGA-SIM-II的三种指标结果对比图

    Fig.  10  Comparison chart of three index results of SGA-SIM-II

    图  11  SCNs的三种指标结果对比图

    Fig.  11  Comparison chart of three index results of SCNs

    表  1  数据集信息

    Table  1  Information of datasets

    数据集 训练样本数 测试样本数 特征 类别
    nonlinear function 600 400 1
    回归问题 Abalone 2000 2177 7
    Compactiv 6144 2048 21
    Iris 120 30 4 3
    分类问题 HAR 7352 2947 561 6
    Gesture recognition 3595 1241 54 24
    下载: 导出CSV

    表  2  各模型在不同数据集上的初始参数

    Table  2  Initial parameters of each model on different datasets

    数据集(期望残差$\ell $) IRWNNs (${L_{\max }}$, $\lambda $, ${T_{\max }}$) SGA-SIM (${L_{\max }}$, $\Upsilon $, ${T_{\max }}$) SCNs (${L_{\max }}$, $\Upsilon $, ${T_{\max }}$)
    nonlinear function (0.05) 100, 150, 1 100, 150:10:200, 20 100, 150:10:200, 20
    Abalone (0.16) 100, 0.5, 1 100, 0.5:0.1:10, 20 100, 0.5:0.1:10, 20
    Compactiv (0.15) 200, 0.5, 1 200, 0.5:0.1:10, 20 200, 0.5:0.1:10, 20
    Iris (0.01) 50, 1, 1 50, 1:1:10, 20 50, 1:1:10, 20
    HAR (0.01) 500, 50, 1 500, 1:1:10, 20 500, 1:1:10, 20
    Gesture recognition (0.05) 500, 0.5, 1 500, 0.5:0.5:10, 20 500, 0.5:0.5:10, 20
    下载: 导出CSV

    表  3  数值模拟例子的实验结果

    Table  3  Experimental results of numerical simulation examples

    模型 节点数$(L)$ 建模时间(s) AVE DEV
    IRWNNs 100.0 3.25 0.1060 0.0301
    SGA-SIM-I 79.8 2.70 0.0014 0.0003
    SGA-SIM-II 79.2 0.15 0.0010 0.0002
    SCNs 79.3 2.93 0.0014 0.0003
    下载: 导出CSV

    表  4  公共数据集的实验结果

    Table  4  Experimental results of public datasets

    数据集 模型 节点数$(L)$ 建模时
    间(s)
    训练
    误差
    测试
    误差
    Abalone IRWNNs 100 0.2543 0.2209 0.2178
    SGA-SIM-I 0.8190 0.1479 0.1763
    SGA-SIM-II 0.2024 0.1446 0.1727
    SCNs 0.8876 0.1477 0.1723
    Compactiv IRWNNs 200 1.3461 0.2720 0.2715
    SGA-SIM-I 5.3443 0.0573 0.0695
    SGA-SIM-II 1.8827 0.0571 0.0670
    SCNs 5.7920 0.0573 0.0695
    Iris IRWNNs 50 0.0169 0.0222 0.0556
    SGA-SIM-I 0.0794 0.0167 0.0333
    SGA-SIM-II 0.0616 0.0167 0.0333
    SCNs 0.0918 0.0173 0.0333
    HAR IRWNNs 500 50.0381 0.0739 0.1233
    SGA-SIM-I 110.7238 0.0147 0.0450
    SGA-SIM-II 27.4965 0.0140 0.0441
    SCNs 111.3737 0.0160 0.0550
    下载: 导出CSV

    表  5  智能手套传感器描述

    Table  5  Smart gloves sensor description

    传感器 描述
    加速度传感器 1)加速度传感器是一种能够测量加速度的传感器, 其能感受加速度并转换成可用输出信号;
    2)数据主要来自 $x$, $y$, $z$ 三个轴.
    陀螺仪传感器 1)陀螺仪通过测量物体运动时的角速度来计算物体旋转的角度和方向;
    2)数据主要来自 $x$, $y$, $z$ 三个轴.
    弯曲传感器 1)弯曲传感器通过阻值将弯曲程度数字化;
    2)弯曲传感器能够测量的弯曲范围为$\left[{{{1}^ \circ },{{180}^ \circ }} \right]$.
    下载: 导出CSV

    表  6  手势识别结果

    Table  6  Gesture recognition result

    算法 建模时间(s) 测试精度 节点数
    IRWNNs 67.84 81.92% 500
    SGA-SIM-I 110.80 94.49% 500
    SGA-SIM-II 15.26 95.19% 500
    SCNs 113.08 94.49% 500
    下载: 导出CSV
  • [1] Wang X, Chen H, Gan C X, Lin H G, Dou Q, Tsougenis E, et al. Weakly supervised deep learning for whole slide lung cancer image analysis. IEEE Transactions on Cybernetics, 2020, 50(9): 3950-3962 doi: 10.1109/TCYB.2019.2935141
    [2] Chai L, Du J, liu Q F, Lee C H. A cross-entropy-guided measure (CEGM) for assessing speech recognition performance and optimizing DNN-based speech enhancement. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021, 29: 106-117 doi: 10.1109/TASLP.2020.3036783
    [3] Wang N, Er M J, Han M. Generalized single-hidden layer feedforward networks for regression problems. IEEE Transactions on Neural Networks and Learning Systems, 2015, 26(6): 1161-1176 doi: 10.1109/TNNLS.2014.2334366
    [4] Ginanjar R R, Kim D S. Real-time SLFN-based node localization using UAV. In: Proceedings of the IEEE International Conference on Industrial Cyber Physical Systems (ICPS). Taiwan, China: IEEE, 2019. 101−106
    [5] Cao, W P, Wang X Z, Zhong M, Gao J Z. A review on neural networks with random weights. Neurocomputing, 2018, 275: 278-287 doi: 10.1016/j.neucom.2017.08.040
    [6] Broomhead D S, Lowe D. Multivariable functional interpolation and adaptive networks. Complex System, 1988, 2: 321-355
    [7] Igelnik B, Pao Y H. Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Transactions on Neural Networks, 1955, 6(6): 1320-1329
    [8] Pao Y H, Takefuji Y. Functional-link net computing: Theory, system architecture, and functionalities. Computer, 1992, 25(5): 76-79 doi: 10.1109/2.144401
    [9] Schmidt W F, Kraaijveld M A, Duin R P W. Feedforward neural networks with random weights. In: Proceedings of the 1st IAPR International Conference on Pattern Recognition. Hague, Netherlands: IEEE, 1992. 1−4
    [10] Husmeier D. Neural Networks for Conditional Probability Estimation: Forecasting Beyond Point Predictions. New York: Springer, 2012.
    [11] Kwok T Y, Yeung D Y. Objective functions for training new hidden units in constructive neural networks. IEEE Transactions on Neural Networks. 1997, 8(5): 1131-1148 doi: 10.1109/72.623214
    [12] Lauret P, Fock E, Mara T A. A node pruning algorithm based on a fourier amplitude sensitivity test method. IEEE Transactions on Neural Networks, 2006, 17(2): 273-293
    [13] Ainsworth T L, Wang Y T, Lee J S. Model-based polarimetric SAR decomposition: an L1 regularization approach. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-13
    [14] Lin M B, Ji R R, Li S J. Network pruning using adaptive exemplar filters. IEEE Transactions on Neural Networks and Learning Systems, 2006, 17(2): 273-293
    [15] Amari S, Murata N. Asymptotic statistical theory of overtraining and cross-validation. IEEE Transactions on Neural Networks, 1997, 8(5): 985-996 doi: 10.1109/72.623200
    [16] Hanson S J, Pratt L Y. Comparing biases for minimal network construction with back-propagation. In: Proceedings of the 1st International Conference on Neural Information Processing Systems. Colorado, USA: MIT Press, 1988. 177−185
    [17] Liu D, Chang T, Zhang Y. A constructive algorithm for feedforward neural networks with incremental training. IEEE Transactions on Circuits & Systems Part I Fundamental Theory & Applications, 2002, 49: 1876-1879
    [18] Zhou P, Jiang Y, Wen C. Data modeling for quality prediction using improved orthogonal incremental random vector functional-link networks. Neurocomputing, 2019, 365: 1-9 doi: 10.1016/j.neucom.2019.06.062
    [19] Qiu X H, Suganthan P N, Amaratunga A J G. Ensemble incremental random vector functional link network for short-term crude oil price forecasting. In: Proceedings of the IEEE Symposium Series on Computational Intelligence (SSCI). Bengaluru, India: IEEE, 2018. 1758−1763
    [20] Li M, Wang D H. Insights into randomized algorithms for neural networks: Practical issues and common pitfalls. Information Sciences, 2017, 382: 170-178
    [21] Dudek G. A method of generating random weights and biases in feedforward neural networks with random hidden nodes. Information Sciences, 2019, 481: 33-56 doi: 10.1016/j.ins.2018.12.063
    [22] Tyukin I Y, Prokhorov D V. Feasibility of random basis function approximators for modeling and control. In: Proceedings of the IEEE Control Applications, (CCA) & Intelligent Control, (ISIC). St. Petersburg, Russia: IEEE, 2009. 1391−1396
    [23] Wang D H, Li M. Stochastic configuration networks: Fundamentals and algorithms. IEEE Transactions on Cybernetics, 2017, 47(10): 3466-3479 doi: 10.1109/TCYB.2017.2734043
    [24] Li M, Huang C, Wang D H. Robust stochastic configuration networks with maximum correntropy criterion for uncertain data regression. Information Sciences, 2018, 473: 73-86
    [25] 代伟, 李德鹏, 杨春雨, 马小平. 一种随机配置网络的模型与数据混合并行学习方法. 自动化学报, 2021, 47(10): 2427-2437 doi: 10.16383/j.aas.c190411

    Dai Wei, Li De-Peng, Yang Chun-Yu, Ma Xiao-Ping. A model and data hybrid parallel learning method for stochastic configuration networks. Acta Automatica Sinica, 2021, 47(10): 2427-2437 doi: 10.16383/j.aas.c190411
    [26] Dai W, Li D P, Zhou P. Stochastic configuration networks with block increments for data modeling in process industries. Information Sciences, 2019, 484: 367-386 doi: 10.1016/j.ins.2019.01.062
    [27] Chen C L P, Liu Z L. Broad learning system: An effective and efficient incremental learning system without the need for deep architecture. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(1): 10-24 doi: 10.1109/TNNLS.2017.2716952
    [28] Chu F, Liang T, Chen C L P, Wang X, Ma X. Weighted broad learning system and its application in nonlinear industrial process modeling. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(8): 3017-3031 doi: 10.1109/TNNLS.2019.2935033
    [29] Feng S, Chen C L P. Fuzzy broad learning system: A novel neuro-fuzzy model for regression and classification. IEEE Transactions on Cybernetics, 2020, 50(2): 414-424 doi: 10.1109/TCYB.2018.2857815
    [30] Wang X H, Zhang T, Xu X M, Chen L, Xing X F, Chen C L P. EEG emotion recognition using dynamical graph convolutional neural networks and broad learning system. In: Proceedings of the IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Madrid, Spain: IEEE, 2018. 1240−1244
    [31] Albert A. Regression and the Moore-Penrose Pseudoinverse. New York: Academic Press, 1972.
    [32] Howland P, Park H. Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(8): 995-1006 doi: 10.1109/TPAMI.2004.46
    [33] Ben-Israel A, Greville T. Generalized Inverses: Theory and Applications. New York: John Wiley and Sons, 1974.
    [34] Anguita D, Ghio A, Oneto L, Parra X, Reyes-Ortiz J L. Human activity recognition on smartphones using a multiclass hardware-friendly support vector machine. In: Proceedings of the International Workshop on Ambient Assisted Living. Vitoria-Gasteiz, Spain: Springer, 2012. 216−223
    [35] Fdez J A, Fernandez A, Luengo J. KEEL data-mining software tool: Data set repository. Integration of Algorithms and Experimental Analysis Framework, Journal of Multiple-Valued Logic & Soft Computing, 2011, 17(2-3): 255-287
    [36] Cheng L, Liu Y, Hou Z G. A rapid spiking neural network approach with an application on hand gesture recognition. IEEE Transactions on Cognitive and Developmental Systems, 2019, 99: 151-161
    [37] Hong C, Lu Y, Liu Z. Survey on 3D hand gesture recognition. IEEE Transactions on Circuits and Systems for Video Technology, 2016, 26(9): 1659-1673 doi: 10.1109/TCSVT.2015.2469551
  • 加载中
图(11) / 表(6)
计量
  • 文章访问数:  340
  • HTML全文浏览量:  90
  • PDF下载量:  139
  • 被引次数: 0
出版历程
  • 收稿日期:  2021-11-03
  • 录用日期:  2022-09-26
  • 网络出版日期:  2022-11-27
  • 刊出日期:  2023-06-20

目录

    /

    返回文章
    返回