Quantum Gate Elman Neural Network and Its Quantized Extended Gradient Back-propagation Training Algorithm
-
摘要: 针对Elman神经网络的学习速度和泛化性能, 提出一种具有量子门结构的新型Elman神经网络模型及其梯度扩展反向传播(Back-propagation)学习算法, 新模型由量子比特神经元和经典神经元构成. 新网络结构采用量子映射层以确保来自上下文单元的局部反馈与隐藏层输入之间的模式一致; 通过量子比特神经元输出与相关量子门参数的修正互补关系以提高网络更新动力. 新学习算法采用搜索然后收敛的策略自适应地调整学习率参数以提高网络学习速度; 通过将上下文单元的权值扩展到隐藏层的权值矩阵, 使其在与隐藏层权值同步更新过程中获取时间序列的额外信息, 从而提高网络上下文单元输出与隐藏层输入之间的匹配程度. 以峰值检波为例的数值实验结果显示, 在量子反向传播学习过程中, 量子门Elman神经网络具有较快的学习速度和良好的泛化性能.Abstract: A novel Elman neural model with hybrid quantum gate structure and a quantized extended-gradient backpropagation (BP) training algorithm are proposed for improving the performance of the conventional Elman network. The novel model is comprised of qubit neurons and classical neurons. The quantum map layer is employed to address the pattern mismatch between the context layer and the input layer. The complementary relationships between the outputs of qubit neurons and the quantum gate parameters are applied to improve the updated ability of the conventional Elman network. The learning rate is adaptively adjusted by the searching and convergent learning strategy, which makes the new neural model achieving convergence with high speed. The context-layer weights are extended into the hidden-layer weights matrix for obtaining the extra gradient information, such that the context-layer patterns match the input-layer patterns with high level. The numerical experiments are carried out to verify the theoretical results and clearly show that the hybrid quantized Elman network using quantized training offers a good performance in terms of both weight convergence and generalization ability.
-
Key words:
- Qubit neuron /
- Elman neural network /
- extended-gradient /
- back-propagation (BP)
-
[1] Elman J L. Finding structure in time. Cognitive Science, 1990, 14(2): 179-211 [2] Jordan M I. Serial Order: A Parallel Distributed Processing Approach, Institute for Cognitive Science Report 8604, University of California, San Diego, 1986 [3] Liou C Y, Huang J C, Yang W C. Modeling word perception using the Elman network. Neurocomputing, 2008, 71(16-18): 3150-3157 [4] Chen S Y, Lin F J, Shyu K K. Direct decentralized neural control for nonlinear MIMO magnetic levitation system. Neurocomputing, 2009, 72(13-15): 3220-3230 [5] Qi Wei-Min, Cheng Yuan-Chu, Ji Qiao-Ling, Cai Wei-You. PID Elman neural network and its application to dynamical system identification. Control and Decision, 2005, 20(10): 1197-1200 (漆为民, 程远楚, 姬巧玲, 蔡维由. PID型Elman网络及在动态系统辨识中的应用研究. 控制与决策, 2005, 20(10): 1197-1200) [6] Song Chong-Zhi, Wu Yu-Guo, Wang Lu, Xie Neng-Gang. Research on fault diagnosis of engine gearbox based on modified Elman neural network. Chinese Journal of Scientific Instrument, 2008, 29(7): 1414-1417 (宋崇智, 吴玉国, 王璐, 谢能刚. 改进Elman网络在发动机齿轮箱故障诊断中的研究. 仪器仪表学报, 2008, 29(7): 1414-1417) [7] Yang Jin, Wen Yu-Mei, Li Ping. Feature extraction and identification of leak acoustic signal in water distribution pipelines using correlation analysis and approximate entropy. Chinese Journal of Scientific Instrument, 2009, 30(2): 272-279 (杨进, 文玉梅, 李平. 基于相关分析和近似熵的管道泄漏声信号特征提取及辨识方法. 仪器仪表学报, 2009, 30(2): 272-279) [8] Cui A Q, Xu H, Jia P F. An Elman neural network-based model for predicting anti-germ performances and ingredient levels with limited experimental data. Expert Systems with Applications, 2011, 38(7): 8186-8192 [9] Li Zu-Xin, Wang Wan-Liang, Hu Wen-Jun. Least squares support vector machines based predictive feedback scheduling for resource-constrained networks. Control and Decision, 2010, 25(3): 361-366 (李祖欣, 王万良, 胡文军. 资源约束网络中基于LSSVM的预测反馈调度. 控制与决策, 2010, 25(3): 361-366) [10] Subrahmanya N, Shin Y C. Constructive training of recurrent neural networks using hybrid optimization. Neurocomputing, 2010, 73(13-15): 2624-2631 [11] Song Q. On the weight convergence of Elman networks. IEEE Transactions on Neural Networks, 2010, 21(3): 463-480 [12] Wang D L, Liu X M, Ahalt S C. On temporal generalization of simple recurrent networks. Neural Networks, 1996, 9(7): 1099-1118 [13] Li X, Chen G R, Chen Z Q, Yuan Z Z. Chaotifying linear Elman networks. IEEE Transactions on Neural Networks, 2002, 13(5): 1193-1199 [14] Toqeer R S, Bayindir N S. Speed estimation of an induction motor using Elman neural network. Neurocomputing, 2003, 55(3-4): 727-730 [15] Kouda N, Matsui N, Nishimura H, Peper F. An examination of qubit neural network in controlling an inverted pendulum. Neural Processing Letters, 2005, 22(3): 277-290 [16] Matsui N, Takai M, Nishimura H. A network model based on qubitlike neuron corresponding to quantum circuit. Electronics and Communications in Japan (Part III: Fundamental Electronic Science), 2000, 83(10): 67-73 [17] Purushothaman G, Karayiannis N B. Quantum neural networks (QNN's): inherently fuzzy feedforward neural networks. IEEE Transactions on Neural Networks, 1997, 8(3): 679-693 [18] Luitel B, Venayagamoorthy G K. Quantum inspired PSO for the optimization of simultaneous recurrent neural networks as MIMO learning systems. Neural Networks, 2010, 23(5): 583-586 [19] Elman J L. Learning and development in neural networks: the importance of starting small. Cognitive Science, 1993, 48(1): 71-99
点击查看大图
计量
- 文章访问数: 1752
- HTML全文浏览量: 90
- PDF下载量: 1187
- 被引次数: 0