一种有效的储备池在线稀疏学习算法

韩敏; 王新迎

doi:10.3724/SP.J.1004.2011.01536

一种有效的储备池在线稀疏学习算法

doi: 10.3724/SP.J.1004.2011.01536 cstr: 32138.14.SP.J.1004.2011.01536

韩敏,
王新迎^,

1.
大连理工大学电子信息与电气工程学部大连 116023

详细信息

通讯作者:
王新迎大连理工大学电子信息与电气工程学部博士研究生.主要研究方向为神经网络,时间序列预测. E-mail: xinying@mail.dlut.edu.cn

计量
- 文章访问数: 2471
- HTML全文浏览量: 71
- PDF下载量: 973
- 被引次数: 0
出版历程
- 收稿日期: 2011-02-24
- 修回日期: 2011-07-07
- 刊出日期: 2011-12-20

An Effective Online Sparse Learning Algorithm for Echo State Networks

HAN Min,
WANG Xin-Ying^,

1.
Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian 116023

摘要

摘要: 为克服传统储备池方法缺乏良好在线学习算法的问题, 同时考虑到储备池本身存在的不适定问题, 本文提出一种储备池在线稀疏学习算法, 对储备池目标函数施加L1正则化约束,并采用截断梯度算法在线近似求解.所提算法在对储备池输出权值进行在线调整的同时, 可对储备池输出权值的稀疏性进行有效控制, 有效保证了网络的泛化性能.理论分析和仿真实例证明所提算法的有效性.
- 递归网络 /
- 回声状态网络 /
- 稀疏 /
- 在线 /
- 优化
Abstract: In order to overcome the lack of effective online learning method for echo state networks and to solve the ill-posed problem of reservoir, an effective online sparse learning algorithm is proposed for echo state networks in this paper. An L1 regularization constraint is added to the objective function of reservoir, and a truncated gradient algorithm is used to approximately solve the problem online. The proposed method can adjust the output weights of reservoir online, control the sparsity of the output weights, and ensure the generalization performance.Theoretical analysis and simulation results demonstrate the effectiveness of the algorithm.
- Recurrent neural networks /
- echo state networks (ESNs) /
- sparse /
- online /
- optimization

HTML全文

参考文献(1)

[1]

Jaeger H, Haas H. Harnessing nonlinearity: predicting chaotic systems and saving energy in wireless communication. Science, 2004, 304(5667): 78-80[2] Maass W, Natschlager T, Markram H. Real-time computing without stable states: a new framework for neural computation based on perturbations. Neural Computation, 2002, 14(11): 2531-2560[3] Han Min, Wang Ya-Nan. Multivariate time series online predictor with Kalman filter trained reservoir. Acta Automatica Sinica, 2010, 36(1): 169-173 (韩敏, 王亚楠. 基于Kalman滤波的储备池多元时间序列在线预报器. 自动化学报, 2010, 36(1): 169-173)[4] Jaeger H, Maass W, Principe J. Special issue on echo state networks and liquid state machines. Neural Networks, 2007, 20(3): 287-289[5] Liu Ying, Zhao Jun, Wang Wei, Wu Yi-Ping, Chen Wei-Chang. Improved echo state network based on data-driven and its application to prediction of blast furnace gas output. Acta Automatica Sinica, 2009, 35(6): 731-738 (刘颖, 赵珺, 王伟, 吴毅平,陈伟昌. 基于数据的改进回声状态网络在高炉煤气发生量预测中的应用. 自动化学报, 2009, 35(6): 731-738)[6] Roseschies B, Igel C. Structure optimization of reservoir networks. Logic Journal of the IGPL, 2010, 18(5): 635-669[7] Shi Z W, Han M. Support vector echo-state machine for chaotic time-series prediction. IEEE Transactions on Neural Networks, 2007, 18(2): 359-372[8] Steil J J. Online stability of backpropagation-decorrela-tion recurrent learning. Neurocomputing, 2006,69(7-9): 642-650[9] Jaeger H. Reservoir riddles: suggestions for echo state network research. In: Proceedings of the IEEE International Joint Conference on Neural Networks. Montreal, Canada: IEEE, 2005. 1460-1462[10] Tibshirani R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B (Methodological), 1996, 58(1): 267-288[11] Shevade S K, Keerthi S S. A simple and efficient algorithm for gene selection using sparse logistic regression. Bioinformatics, 2003, 19(17): 2246-2253[12] Langford J, Li L, Zhang T. Sparse online learning via truncated gradient. The Journal of Machine Learning Research, 2009,10: 777-801[13] Donoho D L. For most large underdetermined systems of linear equations the minimal L1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 2006,59(6): 797-829

施引文献

资源附件(0)

访问统计