2.845

2023影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于一般二阶混合矩的高斯分布估计算法

任志刚 梁永胜 张爱民 庞蓓

任志刚, 梁永胜, 张爱民, 庞蓓. 基于一般二阶混合矩的高斯分布估计算法. 自动化学报, 2018, 44(4): 635-645. doi: 10.16383/j.aas.2017.c160739
引用本文: 任志刚, 梁永胜, 张爱民, 庞蓓. 基于一般二阶混合矩的高斯分布估计算法. 自动化学报, 2018, 44(4): 635-645. doi: 10.16383/j.aas.2017.c160739
REN Zhi-Gang, LIANG Yong-Sheng, ZHANG Ai-Min, PANG Bei. A Gaussian Estimation of Distribution Algorithm Using General Second-order Mixed Moment. ACTA AUTOMATICA SINICA, 2018, 44(4): 635-645. doi: 10.16383/j.aas.2017.c160739
Citation: REN Zhi-Gang, LIANG Yong-Sheng, ZHANG Ai-Min, PANG Bei. A Gaussian Estimation of Distribution Algorithm Using General Second-order Mixed Moment. ACTA AUTOMATICA SINICA, 2018, 44(4): 635-645. doi: 10.16383/j.aas.2017.c160739

基于一般二阶混合矩的高斯分布估计算法

doi: 10.16383/j.aas.2017.c160739
基金项目: 

中国博士后科学基金 2014M560784

国家自然科学基金 61105126

中国博士后科学基金 2016T90922

详细信息
    作者简介:

    梁永胜  西安交通大学电信学院自动化系硕士研究生.2015年获得西安交通大学电信学院自动化系学士学位.主要研究方向为机器学习与智能计算.E-mail:liangyongsheng@stu.xjtu.edu.cn

    张爱民  西安交通大学电信学院自动化系教授.主要研究方向为控制理论与控制工程, 复杂系统的建模、优化与控制.E-mail:zhangam@mail.xjtu.edu.cn

    庞蓓  西安交通大学电信学院自动化系硕士研究生.2016年获得西安交通大学电信学院自动化系学士学位.主要研究方向为机器学习与智能计算.E-mail:beibei@stu.xjtu.edu.cn

    通讯作者:

    任志刚  西安交通大学电信学院自动化系副教授. 2010年获得西安交通大学工学博士学位.主要研究方向为机器学习与智能计算, 复杂系统的建模、优化与控制.本文通信作者.E-mail:renzg@mail.xjtu.edu.cn

A Gaussian Estimation of Distribution Algorithm Using General Second-order Mixed Moment

Funds: 

Postdoctoral Science Foundation of China 2014M560784

National Natural Science Foundation of China 61105126

Postdoctoral Science Foundation of China 2016T90922

More Information
    Author Bio:

      Master student in the Department of Automation Science and Technology, School of Electronic and Information Engineering, Xi$'$an Jiaotong University. He received his bachelor degree from Xi$'$an Jiaotong University in 2015. His research interest covers machine learning and intelligent computing

     Professor in the Department of Automation Science and Technology, School of Electronic and Information Engineering, Xi$'$an Jiaotong University. Her research interest covers control theory and control engineering, modeling, optimization and control of complex systems

      Master student in the Department of Automation Science and Technology, School of Electronic and Information Engineering, Xi$'$an Jiaotong University. She received her bachelor degree from Xi$'$an Jiaotong University in 2016. Her research interest covers machine learning and intelligent computing

    Corresponding author: REN Zhi-Gang   Associate professor in the Department of Automation Science and Technology, School of Electronic and Information Engineering, Xi$'$an Jiaotong University. He received his Ph. D. degree from Xi$'$an Jiaotong University in 2010. His research interest covers machine learning and intelligent computing, modeling, optimization and control of complex systems. Corresponding author of this paper
  • 摘要: 针对传统高斯分布估计算法(Gaussian estimation of distribution algorithms,GEDAs)中变量方差减小速度快、概率密度椭球体(Probability density ellipsoid,PDE)的长轴与目标函数的改进方向相垂直,从而导致算法搜索效率低、容易早熟收敛这一问题,提出一种基于一般二阶混合矩的高斯分布估计算法.该算法利用加权的优秀样本预估高斯均值,并根据沿目标函数的改进方向偏移后的均值来估计协方差矩阵.理论和数值分析表明,这一简单操作可以在不增大算法计算量的前提下自适应地调整概率密度椭球体的位置、大小和长轴方向,提高算法的搜索效率.在14个标准函数上对所提算法进行了测试,由统计出的Cohen's d效应量指标可知该算法的全局寻优能力强于传统高斯分布估计算法;与当前先进的粒子群算法、差分进化算法相比,所提算法可以在相同的函数评价次数内获得9个函数的显著优解.
    1)  本文责任编委 刘艳军
  • 图  1  传统GEDA中PDE的变化示意图

    Fig.  1  Schematic for the change of PDE of traditional GEDA

    图  2  GSM-GEDA的性能随$\eta_{f}$的变化情况

    Fig.  2  The performance variation of GSM-GEDA with regard to $\eta_{f}$

    图  3  长轴长度随迭代次数的变化情况

    Fig.  3  The variation of long axis length with regard to iteration times

    图  4  长轴与目标函数改进方向之间的夹角余弦随迭代次数的变化情况

    Fig.  4  The variation of the cosine value of the angle between long axis and $\hat{\boldsymbol{\delta}}$ with regard to iteration times

    图  5  长轴与最速下降方向之间的夹角余弦随迭代次数的变化情况

    Fig.  5  The variation of the cosine value of the angle between long axis and the steepest descent direction with regard to iteration times

    图  6  函数误差值随迭代次数的变化情况

    Fig.  6  The variation of the function error value with regard to iteration times

    表  1  7种算法最终求得的函数误差值的均值和标准差

    Table  1  The mean and standard deviation of the function error values obtained by 7 algorithms

    函数EMNA$_{g}$AMaLGaMCMA-ESCLPSOCoBiDEMPEDEGSM-GEDA
    $f_{1}$ 2.21E+04$^{-}$
    1.65E+03
    1.63E-14$^{-}$
    8.07E-14
    1.58E-25$^{-}$
    3.35E-26
    0.00E+00$^{+}$
    0.00E+00
    0.00E+00$^{+}$
    0.00E+00
    0.00E+00$^{+}$
    0.00E+00
    3.98E-27
    7.85E-28
    $f_{2}$ 1.10E+04$^{-}$
    1.15E+03
    2.06E-16$^{-}$
    6.64E-16
    1.12E-24$^{-}$
    2.93E-25
    8.40E+02$^{-}$
    1.90E+02
    1.60E-12$^{-}$
    2.90E-12
    1.01E-26$^{+}$
    2.05E-26
    1.78E-26
    4.77E-27
    $f_{3}$ 1.97E+08$^{-}$
    3.18E+07
    3.37E-09$^{-}$
    1.19E-08
    5.54E-21$^{-}$
    1.69E-21
    1.42E+07$^{-}$
    4.19E+06
    7.26E+04$^{-}$
    5.64E+04
    1.01E+01$^{-}$
    8.32E+00
    1.12E-22
    2.85E-23
    $f_{4}$ 7.17E+03$^{-}$
    1.39E+03
    2.17E-11$^{-}$
    1.05E-10
    9.15E+05$^{-}$
    2.16E+06
    6.99E+03$^{-}$
    1.73E+03
    1.16E-03$^{-}$
    2.74E-03
    6.61E-16$^{-}$
    5.68E-16
    1.89E-25
    3.89E-26
    $f_{5}$ 2.81E+04$^{-}$
    2.22E+03
    2.46E+01$^{-}$
    1.05E+01
    2.77E-10$^{-}$
    5.04E-11
    3.86E+03$^{-}$
    4.35E+02
    8.03E+01$^{-}$
    1.51E+02
    7.21E-06$^{-}$
    5.12E-06
    8.90E-11
    3.54E-11
    $f_{6}$ 1.79E+09$^{-}$
    3.70E+08
    1.05E+01$^{+}$
    1.49E+00
    4.78E-01$^{+}$
    1.32E+00
    4.16E+00$^{+}$
    3.48E+00
    4.13E-02$^{+}$
    9.21E-02
    9.65E+00$^{+}$
    4.65E+00
    1.75E+01
    6.09E-01
    $f_{7}$ 1.08E+04$^{-}$
    2.88E+02
    1.79E-15$^{-}$
    6.60E-15
    1.82E-03$^{-}$
    4.33E-03
    4.51E-01$^{-}$
    8.47E-02
    1.77E-03$^{-}$
    3.73E-03
    2.36E-03$^{-}$
    1.15E-03
    0.00E+00
    0.00E+00
    $f_{8}$ 2.10E+01$^{-}$
    3.79E-02
    2.09E+01$^{\approx}$
    5.43E-02
    2.03E+01$^{+}$
    5.72E-01
    2.09E+01$^{\approx}$
    4.41E-02
    2.07E+01$^{+}$
    3.75E-01
    2.09E+01$^{\approx}$
    5.87E-01
    2.09E+01
    5.79E-02
    $f_{9}$ 5.46E+01$^{-}$
    8.62E+00
    3.66E+00$^{+}$
    1.48E+00
    4.45E+02$^{-}$
    7.12E+01
    0.00E+00$^{+}$
    0.00E+00
    0.00E+00$^{+}$
    0.00E+00
    0.00E+00$^{+}$
    0.00E+00
    7.48E+00
    1.93E+00
    $f_{10}$ 1.26E+02$^{-}$
    1.23E+01
    1.91E+00$^{+}$
    1.52E+00
    4.63E+01$^{-}$
    1.16E+01
    1.04E+02$^{-}$
    1.53E+01
    4.41E+01$^{-}$
    1.29E+01
    1.52E+01$^{-}$
    2.98E+00
    6.61E+00
    2.37E+00
    $f_{11}$ 6.25E+00$^{-}$
    1.37E+00
    3.58E-02$^{-}$
    1.29E-01
    7.11E+00$^{-}$
    2.14E+00
    2.60E+01$^{-}$
    1.63E+00
    5.62E+00$^{-}$
    2.19E+00
    2.58E+01$^{-}$
    3.11E+00
    0.00E+00
    0.00E+00
    $f_{12}$ 4.38E+04$^{-}$
    1.54E+04
    1.06E+03$^{\approx}$
    1.87E+03
    1.26E+04$^{-}$
    1.74E+04
    1.79E+04$^{-}$
    5.24E+03
    2.94E+03$^{-}$
    3.93E+03
    1.17E+03$^{-}$
    8.66E+02
    9.45E+02
    1.27E+03
    $f_{13}$ 3.85E+00$^{-}$
    6.32E-01
    2.85E+00$^{-}$
    4.61E-01
    3.43E+00$^{-}$
    7.60E-01
    2.06E+00$^{+}$
    2.15E-01
    2.64E+00$^{\approx}$
    1.13E+00
    2.92E+00$^{-}$
    6.33E-01
    2.77E+00
    3.05E-01
    $f_{14}$ 1.15E+01$^{-}$
    3.53E-01
    1.13E+01$^{-}$
    3.23E-01
    1.47E+01$^{-}$
    3.31E-01
    1.28E+01$^{-}$
    2.48E-01
    1.23E+01$^{-}$
    4.90E-01
    1.23E+01$^{-}$
    4.22E-01
    1.05E+01
    2.85E-01
    $+/{\approx}/-$
    (个数)
    0 / 0 / 143 / 2/ 92 / 0/ 124 / 1 / 94 / 1 / 94 / 1 / 9-
    下载: 导出CSV
  • [1] Larrañnaga P, Lozano J A. Estimation of Distribution Algorithms:A New Tool for Evolutionary Computation. New York, USA:Springer, 2001. 57-100
    [2] Hauschild M, Pelikan M. An introduction and survey of estimation of distribution algorithms. Swarm and Evolutionary Computation, 2011, 1(3):111-128 doi: 10.1016/j.swevo.2011.08.003
    [3] 王圣尧, 王凌, 方晨, 许烨.分布估计算法研究进展.控制与决策, 2012, 27(7):961-966, 974 http://www.cnki.com.cn/Article/CJFDTOTAL-KZYC201207002.htm

    Wang Sheng-Yao, Wang Ling, Fang Chen, Xu Ye. Advances in estimation of distribution algorithms. Control and Decision, 2012, 27(7):961-966, 974 http://www.cnki.com.cn/Article/CJFDTOTAL-KZYC201207002.htm
    [4] Larrañaga P, Etxeberria R, Lozano J A, Peña J M. Optimization in continuous domains by learning and simulation of Gaussian networks. In: Proceedings of 2000 Genetic and Evolutionary Computation Conference. Las Vegas, USA: Morgan Kaufmann, 2000. 201-204
    [5] Lu Q, Yao X. Clustering and learning Gaussian distribution for continuous optimization. IEEE Transactions on Systems, Man, and Cybernetics, Part C:(Applications and Reviews), 2005, 35(2):195-204 doi: 10.1109/TSMCC.2004.841914
    [6] Yuan B, Gallagher M. On the importance of diversity maintenance in estimation of distribution algorithms. In: Proceedings of the 7th Annual Conference on Genetic and Evolutionary Computation. New York, USA: ACM, 2005. 719-726
    [7] Pošík P. Preventing premature convergence in a simple EDA via global step size setting. Parallel Problem Solving from Nature——PPSN X. Lecture Notes in Computer Science. Berlin, Heidelberg, Germany: Springer-Verlag, 2008: . 549-558
    [8] Dong W S, Yao X. Unified Eigen analysis on multivariate Gaussian based estimation of distribution algorithms. Information Sciences, 2008, 178(15):3000-3023 doi: 10.1016/j.ins.2008.01.021
    [9] Ocenasek J, Kern S, Hansen N, Koumoutsakos P. A mixed bayesian optimization algorithm with variance adaptation. Parallel Problem Solving from Nature——PPSN VⅢ. Lecture Notes in Computer Science., Berlin, Heidelberg, Germany: Springer-Verlag, 2004. 352-361
    [10] Bosman P A N, Grahl J, Rothlauf F. SDR: a better trigger for adaptive variance scaling in normal EDAs. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation. New York, USA: ACM, 2007. 492-499
    [11] Cai Y P, Sun X M, Xu H, Jia P F. Cross entropy and adaptive variance scaling in continuous EDA. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation. New York, USA: ACM, 2007. 609-616
    [12] Bosman P A N, Grahl J, Thierens D. Enhancing the performance of maximum-likelihood Gaussian EDAs using anticipated mean shift. Parallel Problem Solving from Nature——PPSN X. Lecture Notes in Computer Science. Berlin, Heidelberg, Germany: Springer-Verlag, 2008: . 133-143
    [13] Bosman P A N, Grahl J, Thierens D. Benchmarking parameter-free AMaLGaM on functions with and without noise. Evolutionary Computation, 2013, 21(3):445-469 doi: 10.1162/EVCO_a_00094
    [14] 程玉虎, 王雪松, 郝名林.一种多样性保持的分布估计算法.电子学报, 2010, 38(3):591-597 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=dianzixb201003018

    Cheng Yu-Hu, Wang Xue-Song, Hao Ming-Lin. An estimation of distribution algorithm with diversity preservation. Acta Electronica Sinica, 2010, 38(3):591-597 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=dianzixb201003018
    [15] Karshenas H, Santana R, Bielza C, Larrañaga P. Regularized continuous estimation of distribution algorithms. Applied Soft Computing, 2013, 13(5):2412-2432 doi: 10.1016/j.asoc.2012.11.049
    [16] Yang P, Tang K, Lu X F. Improving estimation of distribution algorithm on multi-modal problems by detecting promising areas. IEEE Transactions on Cybernetics, 2015, 45(8):1438-1449 doi: 10.1109/TCYB.2014.2352411
    [17] Ding N, Zhou S D, Sun Z Q. Histogram-based estimation of distribution algorithm:a competent method for continuous optimization. Journal of Computer Science and Technology, 2008, 23(1):35-43 doi: 10.1007/s11390-008-9108-0
    [18] 张建华, 曾建潮.基于序贯重点采样粒子滤波的分布估计算法.电子学报, 2010, 38(12):2929-2932 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=dianzixb201012039

    Zhang Jian-Hua, Zeng Jian-Chao. Estimation of distribution algorithm based on sequential importance sampling particle filters. Acta Electronica Sinica, 2010, 38(12):2929-2932 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=dianzixb201012039
    [19] 王丽芳, 曾建潮, 洪毅.利用Copula函数估计概率模型并采样的分布估计算法.控制与决策, 2011, 26(9):1333-1337, 1342 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=kzyjc201109009

    Wang Li-Fang, Zeng Jian-Chao, Hong Yi. Estimation of distribution algorithm modeling and sampling by means of Copula. Control and Decision, 2011, 26(9):1333-1337, 1342 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=kzyjc201109009
    [20] Zhou A M, Sun J Y, Zhang Q F. An estimation of distribution algorithm with cheap and expensive local search. IEEE Transactions on Evolutionary Computation, 2015, 19(6):807-822 doi: 10.1109/TEVC.2014.2387433
    [21] 张放, 鲁华祥.利用条件概率和Gibbs抽样技术为分布估计算法构造通用概率模型.控制理论与应用, 2013, 30(3):307-315 http://www.oalib.com/paper/4746991

    Zhang Fang, Lu Hua-Xiang. General stochastic model for algorithm of distribution estimation with conditional probabilities and Gibbs sampling. Control Theory & Applications, 2013, 30(3):307-315 http://www.oalib.com/paper/4746991
    [22] 钟伟才, 刘静, 刘芳, 焦李成.建立在一般结构Gauss网络上的分布估计算法.电子与信息学报, 2005, 27(3):467-470 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=dzkxxk200503032

    Zhong Wei-Cai, Liu Jing, Liu Fang, Jiao Li-Cheng. Estimation of distribution algorithm based on generic Gaussian networks. Journal of Electronics and Information Technology, 2005, 27(3):467-470 http://www.wanfangdata.com.cn/details/detail.do?_type=perio&id=dzkxxk200503032
    [23] 赵虹. n元椭球面的判定及所围椭球体的体积.数学的实践与认识, 2013, 43(10):279-282

    Zhao Hong. Areas (Volumes) of n-dimension ellipsoid by quadratic curve (surface) enclosed. Mathematics in Practice and Theory, 2013, 43(10):279-282
    [24] Bunch J R, Nielsen C P, Sorensen D C. Rank-one modification of the symmetric eigenproblem. Numerische Mathematik, 1978, 31(1):31-48 doi: 10.1007/BF01396012
    [25] 殷庆祥.实对称矩阵的秩1修正的特征反问题.南通工学院学报(自然科学版), 2003, 2(4):11-14 http://www.doc88.com/p-9919700579946.html

    Yin Qing-Xiang. The inverse problem of rank-1 modification of real symmetric matrices. Journal of Nantong Institute of Technology (Natural Science), 2003, 2(4):11-14 http://www.doc88.com/p-9919700579946.html
    [26] 谢平民, 陈图豪.连续型样本协差阵的正定性.中山大学学报(自然科学版), 1990, 29(1):102-104 http://www.cqvip.com/QK/94631X/199001/429070.html

    Xie Ping-Min, Chen Tu-Hao. The positive definiteness of the covariance matrix of continuous sample. Acta Scientiarum Naturalium Universitatis Sunyatseni, 1990, 29(1):102-104 http://www.cqvip.com/QK/94631X/199001/429070.html
    [27] Suganthan P N, Hansen N, Liang J J, Deb K, Chen Y P, Auger A, Tiwari S. Problem Definitions and Evaluation Criteria for the CEC 2005 Special Session on Real Parameter Optimization, Technical Report, Nanyang Technological University, Singapore, 2005.
    [28] Hansen N, Ostermeier A. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 2001, 9(2):159-195
    [29] Liang J J, Qin A K, Suganthan P N, Baskar S. Comprehensive learning particle swarm optimizer for global optimization of multimodal functions. IEEE Transactions on Evolutionary Computation, 2006, 10(3):281-295 doi: 10.1109/TEVC.2005.857610
    [30] Wang Y, Li H X, Huang T W, Li L. Differential evolution based on covariance matrix learning and bimodal distribution parameter setting. Applied Soft Computing, 2014, 18:232-247 doi: 10.1016/j.asoc.2014.01.038
    [31] Wu G H, Mallipeddi R, Suganthan P N, Wang R, Chen H K. Differential evolution with multi-population based ensemble of mutation strategies. Information Sciences, 2016, 329:329-345 doi: 10.1016/j.ins.2015.09.009
    [32] Cohen J. Statistical Power Analysis for the Behavioral Sciences (Second Edition). Hillsdale, NJ: Lawrence Erlbaum Associates, 1988.
  • 加载中
图(6) / 表(1)
计量
  • 文章访问数:  2451
  • HTML全文浏览量:  298
  • PDF下载量:  722
  • 被引次数: 0
出版历程
  • 收稿日期:  2016-10-27
  • 录用日期:  2017-03-30
  • 刊出日期:  2018-04-20

目录

    /

    返回文章
    返回