2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于罚函数内点法的泄露积分型回声状态网的参数优化

伦淑娴 胡海峰

伦淑娴, 胡海峰. 基于罚函数内点法的泄露积分型回声状态网的参数优化. 自动化学报, 2017, 43(7): 1160-1168. doi: 10.16383/j.aas.2017.c160541
引用本文: 伦淑娴, 胡海峰. 基于罚函数内点法的泄露积分型回声状态网的参数优化. 自动化学报, 2017, 43(7): 1160-1168. doi: 10.16383/j.aas.2017.c160541
LUN Shu-Xian, HU Hai-Feng. Parameter Optimization of Leaky Integrator Echo State Network with Internal-point Penalty Function Method. ACTA AUTOMATICA SINICA, 2017, 43(7): 1160-1168. doi: 10.16383/j.aas.2017.c160541
Citation: LUN Shu-Xian, HU Hai-Feng. Parameter Optimization of Leaky Integrator Echo State Network with Internal-point Penalty Function Method. ACTA AUTOMATICA SINICA, 2017, 43(7): 1160-1168. doi: 10.16383/j.aas.2017.c160541

基于罚函数内点法的泄露积分型回声状态网的参数优化

doi: 10.16383/j.aas.2017.c160541
基金项目: 

国家自然科学基金 61573072

2011年辽宁省第一批科学技术计划项目 2011402001

国家自然科学基金 21506014

辽宁省教育厅科学技术研究项目 L2015008

辽宁省自然科学基金 2014020143

详细信息
    作者简介:

    胡海峰 渤海大学硕士研究生.主要研究方向为神经网络.E-mail:15504991421@163.com

    通讯作者:

    伦淑娴 渤海大学教授.2005年获得东北大学控制理论与控制工程专业博士学位.2011年在中国科学院自动化研究所从事博士后研究.主要研究方向为神经网络, 能源互联网, 光伏发电系统建模与能量管理.本文通信作者. E-mail:jzlunzi@163.com

Parameter Optimization of Leaky Integrator Echo State Network with Internal-point Penalty Function Method

Funds: 

National Nature Science Foundation of China 61573072

the First Batch of Science and Technology Projects of Liaoning Province 2011402001

National Nature Science Foundation of China 21506014

Science and Technology Research Projects of the Department of Education of Liaoning Province L2015008

Nature Science Foundation of Liaoning Province 2014020143

More Information
    Author Bio:

      Master student at Bohai University. His main research interest is neural network

    Corresponding author: LUN Shu-Xian  Professor at Bohai University. She received her Ph. D. degree from Northeastern University in 2005. She did her postdoctor research at the Institute of Automation, Chinese Academy of Sciences in 2011. Her research interest covers neural network, energy internet, modeling and energy management for photovoltaic power generation systems. Corresponding author of this paper. E-mail:jzlunzi@163.com
  • 摘要: 为了提升泄露积分型回声状态网(Leaky integrator echo state network,Leaky-ESN)的性能,提出利用罚函数内点法优化Leaky-ESN的全局参数,如泄漏率、内部连接权矩阵谱半径、输入比例因子等,这克服了通过反复试验法选取参数值而降低了Leaky-ESN模型的优越性和性能.Leaky-ESN的全局参数必须保障回声状态网满足回声状态特性,因此它们之间存在不等式约束条件.有学者提出利用随机梯度下降法来优化内部连接权矩阵谱半径、输入比例因子、泄露率三个全局参数,一定程度上提高了Leaky-ESN的逼近精度.然而,随机梯度下降法是解决无约束优化问题的基本算法,在利用随机梯度下降法优化参数时,没有考虑参数必须满足回声特性的约束条件(不等式约束条件),致使得到的参数值不是最优解.由于罚函数内点法可以求解具有不等式约束的最优化问题,应用范围广,收敛速度较快,具有很强的全局寻优能力.因此,本文提出利用罚函数内点法优化Leaky-ESN的全局参数,并以时间序列预测为例,检验优化后的Leaky-ESN的预测性能,仿真结果表明了本文提出方法的有效性.
    1)  本文责任编委 魏庆来
  • 图  1  回声状态网络的拓扑结构

    Fig.  1  Structure of echo state network

    图  2  第一时间序列的训练误差

    Fig.  2  Training errors for the first time series

    图  3  第一时间序列的预测

    Fig.  3  Predicted values for the first time series

    图  4  第一时间序列的预测误差

    Fig.  4  Predicted errors for the first time series

    图  5  第一时间序列预测的NRMSE分布图

    Fig.  5  NRMSE of predicted values for the first time series

    图  6  第二时间序列的训练误差

    Fig.  6  Training errors for the second time series

    图  7  第二时间序列的预测

    Fig.  7  Predicted values for the second time series

    图  8  第二时间序列的预测误差

    Fig.  8  Predicted errors for the second time series

    图  9  第二时间序列预测的NRMSE分布图

    Fig.  9  NRMSE of predicted values for the second time series

    表  1  Leaky-ESN的测试NRMSE

    Table  1  Testing NRMSE of Leaky-ESN

    方法第一时间序列第二时间序列
    随机梯度下降法0.212400.17710
    罚函数内点法0.086980.01397
    下载: 导出CSV
  • [1] Jaeger H. Tutorial on Training Recurrent Neural Networks, Covering BPTT, RTRL, EKF, and the "Echo State Network" Approach. Technical Report GMD Report 159, German National Research Center for Information Technology, German, 2002.
    [2] Jaeger H. The "Echo State" Approach to Analysing and Training Recurrent Neural Networks. Technical Report GMD report 148, German National Research Center for Information Technology, German, 2001.
    [3] Jaeger H, Haas H. Harnessing nonlinearity:predicting chaotic systems and saving energy in wireless communication. Science, 2004, 304(5667):78-80 doi: 10.1126/science.1091277
    [4] Lun S X, Wang S, Guo T T, Du C J. An Ⅰ-Ⅴ model based on time warp invariant echo state network for photovoltaic array with shaded solar cells. Solar Energy, 2014, 105:529-541 doi: 10.1016/j.solener.2014.04.023
    [5] Skowronski M D, Harris J G. Noise-robust automatic speech recognition using a predictive echo state network. IEEE Transactions on Audio, Speech, and Language Processing, 2007, 15(5):1724-1730 doi: 10.1109/TASL.2007.896669
    [6] Han S I, Lee J M. Precise positioning of nonsmooth dynamic systems using fuzzy wavelet echo state networks and dynamic surface sliding mode control. IEEE Transactions on Industrial Electronics, 2013, 60(11):5124-5136 doi: 10.1109/TIE.2012.2218560
    [7] Li G Q, Niu P F, Zhang W P, Zhang Y. Control of discrete chaotic systems based on echo state network modeling with an adaptive noise canceler. Knowledge-Based Systems, 2012, 35(15):35-40 http://www.sciencedirect.com/science/article/pii/S0950705112001153
    [8] Song R Z, Xiao W D, Sun C Y. A new self-learning optimal control laws for a class of discrete-time nonlinear systems based on ESN architecture. Science China Information Sciences, 2014, 57(6):Article No. 068202 doi: 10.1007%2Fs11432-013-4954-y.pdf
    [9] Lun S X, Yao X S, Qi H Y, Hu H F. A novel model of leaky integrator echo state network for time-series prediction. Neurocomputing, 2015, 159(1):58-66 http://linkinghub.elsevier.com/retrieve/pii/S0925231215001782
    [10] Bianchi F M, Scardapane S, Uncini A, Rizzi A, Sadeghian A. Prediction of telephone calls load using echo state network with exogenous variables. Neural Networks, 2015, 71(C):204-213 https://www.researchgate.net/publication/281677244_Prediction_of_telephone_calls_load_using_Echo_State_Network_with_exogenous_variables
    [11] Jaeger H, Lukoševičius M, Popovici D, Siewert U. Optimization and applications of echo state networks with leaky-integrator neurons. Neural Networks, 2007, 20(3):335-352 doi: 10.1016/j.neunet.2007.04.016
    [12] Lukoševičius M. A practical guide to applying echo state networks. Neural Networks:Tricks of the Trade (Second Edition). Berlin Heidelberg:Springer-Verlag, 2012. 659-686
    [13] Nocedal J, Wright S J. Numerical Optimization (Second Edition). New York:Springer, 2006. 30-31
  • 加载中
图(9) / 表(1)
计量
  • 文章访问数:  1698
  • HTML全文浏览量:  122
  • PDF下载量:  714
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-07-22
  • 录用日期:  2016-10-09
  • 刊出日期:  2017-07-20

目录

    /

    返回文章
    返回