2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法

暴琳 孙晓燕 巩敦卫 张勇

暴琳, 孙晓燕, 巩敦卫, 张勇. 融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法. 自动化学报, 2023, 49(10): 2188−2200 doi: 10.16383/j.aas.c200604
引用本文: 暴琳, 孙晓燕, 巩敦卫, 张勇. 融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法. 自动化学报, 2023, 49(10): 2188−2200 doi: 10.16383/j.aas.c200604
Bao Lin, Sun Xiao-Yan, Gong Dun-Wei, Zhang Yong. Enhanced restricted Boltzmann machine-driven interactive estimation of distribution algorithms with attention mechanism. Acta Automatica Sinica, 2023, 49(10): 2188−2200 doi: 10.16383/j.aas.c200604
Citation: Bao Lin, Sun Xiao-Yan, Gong Dun-Wei, Zhang Yong. Enhanced restricted Boltzmann machine-driven interactive estimation of distribution algorithms with attention mechanism. Acta Automatica Sinica, 2023, 49(10): 2188−2200 doi: 10.16383/j.aas.c200604

融合注意力机制的增强受限玻尔兹曼机驱动的交互式分布估计算法

doi: 10.16383/j.aas.c200604
基金项目: 国家自然科学基金(61876184)资助
详细信息
    作者简介:

    暴琳:江苏科技大学电子信息学院讲师. 2020年获中国矿业大学控制理论与控制工程专业博士学位. 主要研究方向为进化计算与机器学习. E-mail: baolin_zj@163.com

    孙晓燕:中国矿业大学信息与控制工程学院教授. 2009年获中国矿业大学控制理论与控制工程专业博士学位. 主要研究方向为进化计算和机器学习. 本文通信作者. E-mail: xysun78@126.com

    巩敦卫:中国矿业大学信息与控制工程学院教授. 1999年获中国矿业大学控制理论与控制工程专业博士学位. 主要研究方向为超目标优化计算智能, 动态不确定性优化, 软件测试、调度、路径规划以及大数据处理与分析. E-mail: dwgong@vip.163.com

    张勇:中国矿业大学信息与控制工程学院教授. 2009年获中国矿业大学控制理论与控制工程专业博士学位. 主要研究方向为群体智能和机器学习. E-mail: yongzh401@126.com

Enhanced Restricted Boltzmann Machine-driven Interactive Estimation of Distribution Algorithms With Attention Mechanism

Funds: Supported by National Natural Science Foundation of China (61876184)
More Information
    Author Bio:

    BAO Lin Lecturer at the School of Electronics and Information, Jiangsu University of Science and Technology. She received her Ph.D. degree in control theory and control engineering from China University of Mining and Technology in 2020. Her research interest covers evolutionary computation and machine learning

    SUN Xiao-Yan Professor at the School of Information and Control Engineering, China University of Mining and Technology. She received her Ph.D. degree in control theory and control engineering from China University of Mining and Technology in 2009. Her research interest covers evolutionary computation and machine learning. Corresponding author of this paper

    GONG Dun-Wei Professor at the School of Information and Control Engineering, China University of Mining and Technology. He received his Ph.D. degree in control theory and control engineering from China University of Mining and Technology in 1999. His research interest covers computation intelligence in many-objective optimization, dynamic and uncertain optimization, software engineering, scheduling, path planning, and big data processing and analysis

    ZHANG Yong Professor at the School of Information and Control Engineering, China University of Mining and Technology. He received his Ph.D. degree in control theory and control engineering from China University of Mining and Technology in 2009. His research interest covers swarm intelligence and machine learning

  • 摘要: 面向用户生成内容(User generated content, UGC)的进化搜索在大数据及个性化服务领域已引起广泛关注, 其关键在于基于多源异构用户生成内容构建用户认知偏好模型, 进而设计高效的进化搜索机制. 针对此, 提出融合注意力机制(Attention mechanism, AM)的受限玻尔兹曼机(Restricted Boltzmann machine, RBM)偏好认知代理模型构建机制, 并应用于交互式分布估计算法(Interactive estimation of distribution algorithm, IEDA), 设计含用户生成内容的个性化进化搜索策略. 基于用户群体提供的文本评论, 以及搜索物品的类别文本, 构建无监督受限玻尔兹曼机模型提取广义特征; 设计注意力机制, 融合广义特征, 获取对用户认知偏好高度相关特征的集成; 利用该特征再次训练受限玻尔兹曼机, 实现对用户偏好认知代理模型的构建; 根据用户偏好认知代理模型, 给出交互式分布估计算法概率更新模型以及物品适应度评价函数, 实现物品个性化进化搜索. 算法在亚马逊个性化搜索实例的应用验证了用户认知偏好模型的可靠性, 以及个性化进化搜索的有效性.
  • 图  1  AM-ERBM-IEDA算法框架

    Fig.  1  The framework of AM-ERBM-IEDA algorithm

    图  2  基于注意力机制和RBM的用户认知偏好模型

    Fig.  2  User cognitive preference model based on attention mechanism and RBM

    图  3  测试用户个性化搜索实验

    Fig.  3  Experimental results of a test user

    图  4  CDs_and_Vinyl数据集测试用户个性化搜索实验

    Fig.  4  Experimental results of a test user on CDs_and_Vinyl

    图  5  Music数据集某用户个性化搜索实验

    Fig.  5  Experimental results of a test user on Music

    图  6  Games数据集某用户个性化搜索实验

    Fig.  6  Experimental results of a test user on Games

    表  1  数据集统计信息

    Table  1  Statistical information of datasets

    数据集# 用户# 项目# 评分
    Digital_Music (Music)478235266414836006
    Video_Games (Games)826767502101324753
    Apps_for_Android (Apps)1323884612752638173
    Kindle_Store (Kindle)14068904305303205467
    CDs_and_Vinyl (CDs)15785974863603749004
    Movies_and_TV (Movies)20886202009414607047
    Yelp19124941803477778794
    下载: 导出CSV

    表  2  算法的实验参数

    Table  2  Experimental parameters of our algorithm

    参数数值
    ${n_1}$类别标签数量
    ${n_2}$200
    $m$0.8 ~ 1.2倍类别标签数量
    学习率0.1
    动量0.5 ~ 0.9
    训练次数20
    $Pop$0.3倍测试数据集规模
    $k$10
    $\alpha $0.3
    $\beta $0.2
    $N$10
    下载: 导出CSV

    表  3  对比实验结果

    Table  3  Experiments compared with popular recommendation algorithms

    算法 评价指标RandomPopularityBPRMFConvMFATRankRBMAEDADRBMRBM-MsHAtRBM-MsH
    MusicRMSE3.1441.8983.1302.1981.2981.2641.2971.221*
    HR0.07650.07930.07640.07420.07780.07840.09240.09060.0951*
    mAP0.7610.7230.8110.7280.7780.8110.8870.8800.879*
    运行时间(s)0.0200.1820.494276.7162.9000.2211.5990.6721.766*
    GamesRMSE3.5161.9733.4972.4821.2851.3321.2711.242*
    HR0.08100.09300.07530.09450.08690.08040.08150.08090.0985*
    mAP0.7470.8730.7070.9150.7850.7360.7600.7770.827*
    运行时间(s)0.0140.1960.402131.7163.0950.1512.3460.7192.785*
    AppsRMSE3.1642.1463.1192.6991.5231.5431.5071.486*
    HR0.07990.07950.08520.07010.08870.07590.07460.07600.0818*
    mAP0.7360.7140.7360.6880.7590.7180.7120.7480.771*
    运行时间(s)0.0140.1700.34490.4892.5730.1030.6460.3881.476*
    KindleRMSE4.3192.2844.3172.2131.4371.5491.4451.168*
    HR0.02980.02220.02780.02210.03010.02860.02950.02970.0308*
    mAP0.9140.9200.8570.8330.9000.8940.8670.8750.926*
    运行时间(s)0.0140.7611.205416.5328.74510.06026.2237.22423.478*
    CDsRMSE4.2182.1824.2172.6941.4821.5341.4321.241*
    HR0.01190.01360.01010.01070.01080.01100.01100.01050.0147*
    mAP0.8470.8250.8260.8170.8440.8450.8520.8380.921*
    运行时间(s)0.0163.8335.406884.51932.3075.34531.38228.11135.836*
    MoviesRMSE3.0681.9603.0292.2711.1911.1851.1671.176*
    HR0.01340.01530.01440.01830.01660.01380.01540.01710.0173*
    mAP0.6680.7690.7020.8380.6820.6720.7660.8000.770*
    运行时间(s)0.0142.1863.261506.12518.3410.4659.4681.81510.978*
    YelpRMSE3.1321.7093.1942.1950.9981.0250.9890.967*
    HR0.01850.02280.01960.02590.02100.02080.02320.02330.0268*
    mAP0.6710.7750.7350.8710.7830.7350.8120.8860.912*
    运行时间(s)0.01765.50427.7294824.915159.44623.32526.66911.32620.551
    下载: 导出CSV

    表  4  测试用户个性化搜索实验结果

    Table  4  Experimental results of a test user

    百分比 (%)测试用户
    RMSEHRAP
    100.8740.004550.876
    200.7660.004590.947
    300.7250.007000.977
    400.6920.009441.000
    500.6800.011681.000
    600.6790.010200.895
    700.6780.014260.924
    800.6440.019610.721
    900.6200.040680.812
    下载: 导出CSV

    表  5  对比实验结果

    Table  5  Comparison of experimental results

    算法评价指标IEDARBMIGARBMAEDADRBMIEDARIEDA-MsHAtRIEDA-MsH
    MusicRMSE1.1601.2041.4800.9550.955
    HR0.01840.02220.02360.02300.02860.0305*
    AP0.6010.8150.8970.9140.9310.956*
    GamesRMSE1.3311.3511.5601.1871.176*
    HR0.02310.02050.02010.02380.02450.0246
    AP0.7100.7640.7870.8700.8790.928*
    AppsRMSE1.5371.5341.6301.5741.572*
    HR0.03300.03250.03240.03500.03510.0354*
    AP0.6390.6570.6380.7510.7360.779*
    KindleRMSE0.9080.9001.0640.7000.711*
    HR0.007560.007580.007700.007600.008740.00888*
    AP0.7520.7830.7400.7330.8340.853*
    CDsRMSE1.4061.4051.5891.3881.386
    HR0.003960.004260.004520.004800.004800.00486*
    AP0.8180.8490.8900.9310.9290.923*
    MoviesRMSE1.2751.2761.2101.1881.132*
    HR0.006900.007380.006960.007420.008400.00851*
    AP0.4850.5390.4990.5260.6300.642*
    YelpRMSE0.7520.7490.8960.7230.746*
    HR0.004690.004690.005900.006460.006980.00970*
    AP0.5160.5820.6370.7020.7540.924*
    下载: 导出CSV
  • [1] 李金忠, 刘关俊, 闫春钢, 蒋昌俊. 排序学习研究进展与展望. 自动化学报, 2018, 44(8): 1345-1369

    Li Jin-Zhong, Liu Guan-Jun, Yan Chun-Gang, Jiang Chang-Jun. Research advances and prospects of learning to rank. Acta Automatica Sinica, 2018, 44(8): 1345-1369
    [2] Batmaz Z, Yurekli A, Bilge A, Kaleli C. A review on deep learning for recommender systems: Challenges and remedies. Artificial Intelligence Review, 2019, 52(1): 1-37 doi: 10.1007/s10462-018-9654-y
    [3] Wang Z K, Liu H Z, Du Y P, Wu Z H, Zhang X. Unified embedding model over heterogeneous information network for personalized recommendation. In: Proceedings of the 28th International Joint Conference on Artificial Intelligence. Macao, China: AAAI Press, 2019. 3813−3819
    [4] 饶子昀, 张毅, 刘俊涛, 曹万华. 应用知识图谱的推荐方法与系统[J]. 自动化学报, 2021, 47(9): 2061-2077

    Rao Zi-Yun, Zhang Yi, Liu Jun-Tao, Cao Wan-Hua. Recommendation methods and systems using knowledge graph. Acta Automatica Sinica, 2021, 47(9): 2061-2077
    [5] Rendle S, Freudenthaler C, Gantner Z, Schmidt-Thieme L. BPR: Bayesian personalized ranking from implicit feedback. In: Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Montreal, Canada: AUAI Press, 2009. 452−461
    [6] He X N, Liao L Z, Zhang H W, Nie L Q, Hu X, Chua T S. Neural collaborative filtering. In: Proceedings of the 26th International Conference on World Wide Web. Perth, Australia: ACM, 2017. 173−182
    [7] Truong Q T, Salah A, Lauw H W. Bilateral variational autoencoder for collaborative filtering. In: Proceedings of the 14th ACM International Conference on Web Search and Data Mining. New York, USA: Association for Computing Machinery, 2021. 292–300
    [8] Bao L, Sun X Y, Chen Y, Gong D W, Zhang Y W. Restricted Boltzmann machine-driven interactive estimation of distribution algorithm for personalized search. Knowledge-Based Systems, 2020, 43: Article No. 106030
    [9] Kim D, Park C, Oh J, Lee S, Yu H. Convolutional matrix factorization for document context-aware recommendation. In: Proceedings of the 10th ACM Conference on Recommender Systems. Boston, USA: ACM, 2016. 233−240
    [10] Jin J R, Qin J R, Fang Y C, Du K, Zhang W N, Yu Y, et al. An efficient neighborhood-based interaction model for recommendation on heterogeneous graph. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Virtual Event, USA: Association for Computing Machinery, 2020. 75−84
    [11] Mnih V, Heess N, Graves A, Kavukcuoglu K. Recurrent models of visual attention. In: Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada: ACM, 2014. 2204−2212
    [12] Zhou C, Bai J Z, Song J S, Liu X F, Zhao Z C, Chen X S, et al. ATRank: An attention-based user behavior modeling framework for recommendation. In: Proceedings of the 32nd AAAI Conference on Artificial Intelligence and 30th Innovative Applications of Artificial Intelligence Conference and 8th AAAI Symposium on Educational Advances in Artificial Intelligence. New Orleans, USA: AAAI, 2018. Article No. 559
    [13] 汤文兵, 任正云, 韩芳. 基于注意力机制的协同卷积动态推荐网络. 自动化学报, 2021, 47(10): 2438-2448 doi: 10.16383/j.aas.c190820

    Tang Wen-Bing, Ren Zheng-Yun, Han Fang. Attention-based collaborative convolutional dynamic network for recommendation. Acta Automatica Sinica, 2021, 47(10): 2438-2448 doi: 10.16383/j.aas.c190820
    [14] Li J C, Wang Y J, McAuley J. Time interval aware self-attention for sequential recommendation. In: Proceedings of the 13th International Conference on Web Search and Data Mining. Houston, USA: ACM, 2020. 322−330
    [15] Sun X Y, Gong D W, Jin Y C, Chen S S. A new surrogate-assisted interactive genetic algorithm with weighted semisupervised learning. IEEE Transactions on Cybernetics, 2013, 43(2): 685-698 doi: 10.1109/TSMCB.2012.2214382
    [16] Zhou G R, Mou N, Fan Y, Pi Q, Bian W J, Zhou C, et al. Deep interest evolution network for click-through rate prediction. In: Proceedings of the 33rd AAAI Conference on Artificial Intelligence and 31st Innovative Applications of Artificial Intelligence Conference and 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Honolulu, USA: AAAI, 2019. Article No. 729
    [17] Chen Y, Jin Y C, Sun X Y. Language model based interactive estimation of distribution algorithm. Knowledge-Based Systems, 2020, 200: Article No. 105980 doi: 10.1016/j.knosys.2020.105980
    [18] Salameh M, Singh S, Li S W, Krishnamurthy M. Surrogate vibration modeling approach for design optimization of electric machines. IEEE Transactions on Transportation Electrification, 2020, 6(3): 1126-1133 doi: 10.1109/TTE.2020.3017232
    [19] Gao Y, Yang T, Bozhko S, Wheeler P, Dragičević T. Filter design and optimization of electromechanical actuation systems using search and surrogate algorithms for more-electric aircraft applications. IEEE Transactions on Transportation Electrification, 2020, 6(4): 1434-1447 doi: 10.1109/TTE.2020.3019729
    [20] Bao L, Sun X Y, Chen Y, Man G Y, Shao H. Restricted Boltzmann machine-assisted estimation of distribution algorithm for complex problems. Complexity, 2018, 2018: 2609014
    [21] Zhao F, Zeng Z, Liu H Q, Lan R, Fan J L. Semisupervised approach to surrogate-assisted multiobjective kernel intuitionistic fuzzy clustering algorithm for color image segmentation. IEEE Transactions on Fuzzy Systems, 2020, 28(6): 1023-1034 doi: 10.1109/TFUZZ.2020.2973121
    [22] Min A T W, Ong Y S, Gupta A, Goh C K. Multiproblem surrogates: Transfer evolutionary multiobjective optimization of computationally expensive problems. IEEE Transactions on Evolutionary Computation, 2019, 23(1): 15-28 doi: 10.1109/TEVC.2017.2783441
    [23] Wang S, Liu J, Jin Y C. Surrogate-assisted robust optimization of large-scale networks based on graph embedding. IEEE Transactions on Evolutionary Computation, 2020, 24(4): 735-749 doi: 10.1109/TEVC.2019.2950935
    [24] Cai X W, Gao L, Li X Y. Efficient generalized surrogate-assisted evolutionary algorithm for high-dimensional expensive problems. IEEE Transactions on Evolutionary Computation, 2020, 24(2): 365-379 doi: 10.1109/TEVC.2019.2919762
    [25] Le Q, Mikolov T. Distributed representations of sentences and documents. In: Proceedings of the 31st International Conference on Machine Learning. Beijing, China: ACM, 2014. 1188−1196
    [26] Hinton G E. Training products of experts by minimizing contrastive divergence. Neural Computation, 2002, 14(8): 1771-1800 doi: 10.1162/089976602760128018
    [27] Feng S, Chen C L P. A fuzzy restricted Boltzmann machine: Novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers. IEEE Transactions on Fuzzy Systems, 2018, 26(1): 117-130 doi: 10.1109/TFUZZ.2016.2639064
  • 加载中
图(6) / 表(5)
计量
  • 文章访问数:  1268
  • HTML全文浏览量:  273
  • PDF下载量:  185
  • 被引次数: 0
出版历程
  • 收稿日期:  2020-07-29
  • 网络出版日期:  2021-06-01
  • 刊出日期:  2023-10-24

目录

    /

    返回文章
    返回