An Norm 1 Regularization Term ELM Algorithm Based on Surrogate Function and Bayesian Framework
-
摘要: 针对极端学习机 (Extreme learning machine, ELM)算法的不适定问题和模型规模控制问题,本文提出基于1范数正则项的改进型ELM算法. 通过在二次损失函数基础上引入1范数正则项以控制模型规模,改善ELM的泛化能力.此外,为简化1范数 正则化方法的求解过程,利用边际优化方法,构建适当的替代函数,以便于采用贝叶斯方法代替计算复杂的 交叉检验方法,并实现正则化参数的自适应估计.仿真结果表明,本文所提算法能够有效简化模型结构,并 保持较高的预测精度.Abstract: Focusing on the ill-posed problem and the model scale control of ELM (Extreme learning machine), this paper proposes an improved ELM algorithm based on 1-norm regularization term. This is achieved by involving an 1-norm regularization term into the original square cost function, and it can be used to control the model scale and enhance the generalization capability. Furthermore, to simplify the solving process of the 1-norm regularization method, the bound optimization algorithm is employed and a suitable surrogate function is established. Based on the surrogate function, the Bayesian algorithm can be used to substitute the complicated cross validation method and estimate the regularization parameter adaptively. Simulation results illustrate that the proposed method can effectively simplify the model structure, while remaining acceptable prediction accurate.
-
[1] Meng Zu-Qiang, Cai Zi-Xing. Identification method of nonlinear systems based on parallel genetic algorithm. Control and Decision, 2003, 18(3): 367-370 (蒙祖强, 蔡自兴. 一种基于并行遗传算法的非线性系统辨识方法. 控制与决策, 2003, 18(3): 367-370)[2] Qiao Jun-Fei, Han Hong-Gui. Optimal structure design for RBFNN structure. Acta Automatica Sinica, 2010, 36(6): 865-872 (乔俊飞, 韩红桂. RBF神经网络的结构动态优化设计. 自动化学报, 2010, 36(6): 865-872)[3] Ye Jian, Ge Lin-Dong, Wu Yue-Xian. An application of improved RBF neural network in modulation recognition. Acta Automatica Sinica, 2007, 33(6): 652-654 (叶健, 葛临东, 吴月娴. 一种优化的RBF神经网络在调制识别中的应用. 自动化学报, 2007, 33(6): 652-654)[4] Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: theory and applications. Neurocomputing, 2006, 70(1-3): 489-501[5] Malathi V, Marimuthu N S, Baskar S. Intelligent approaches using support vector machine and extreme learning machine for transmission line protection. Neurocomputing, 2010, 73(10-12): 2160-2167[6] Minhas R, Baradarani A, Seifzadeh S, Wu Q M J. Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing, 2010, 73(10-12): 1906-1917[7] Tang X L, Han M. Partial Lanczos extreme learning machine for single output regression problems. Neurocomputing, 2009, 72(13-15): 3066-3076[8] Huang G B, Chen L. Enhanced random search based incremental extreme learning machine. Neurocomputing, 2008, 71(16-18): 3460-3468[9] Huang G B, Chen L, Siew C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Network, 2006, 17(4): 879-892[10] Hansen P C. Rank-deficient and Discrete Ill--posed Problems: Numerical Aspects of Linear Inversion. Philadelphia: SIAM, 1998. 45-68[11] Kaban A. On Bayesian classification with Laplace priors. Pattern Recognition Letters, 2007, 28(10): 1271-1282[12] Krishnapuram B, Carin L, Figueiredo M A T, Hartemink A J. Sparse multinomial logistic regression: fast algorithms and generalization bounds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(6): 957-968
点击查看大图
计量
- 文章访问数: 2261
- HTML全文浏览量: 43
- PDF下载量: 1402
- 被引次数: 0