2.765

2022影响因子

(CJCR)

  • 中文核心
  • EI
  • 中国科技核心
  • Scopus
  • CSCD
  • 英国科学文摘

留言板

尊敬的读者、作者、审稿人, 关于本刊的投稿、审稿、编辑和出版的任何问题, 您可以本页添加留言。我们将尽快给您答复。谢谢您的支持!

姓名
邮箱
手机号码
标题
留言内容
验证码

基于强化学习的减少烘丝过程中烟丝 “干头” 量的方法

毕素环 蒋一翔 于树松 丁香乾 牟亮亮 王彬

毕素环, 蒋一翔, 于树松, 丁香乾, 牟亮亮, 王彬. 基于强化学习的减少烘丝过程中烟丝 “干头” 量的方法. 自动化学报, 2023, 49(8): 1679−1687 doi: 10.16383/j.aas.c190367
引用本文: 毕素环, 蒋一翔, 于树松, 丁香乾, 牟亮亮, 王彬. 基于强化学习的减少烘丝过程中烟丝 “干头” 量的方法. 自动化学报, 2023, 49(8): 1679−1687 doi: 10.16383/j.aas.c190367
Bi Su-Huan, Jiang Yi-Xiang, Yu Shu-Song, Ding Xiang-Qian, Mu Liang-Liang, Wang Bin. A method for reducing over-dried tobacco at head stage of drying process based on reinforcement learning. Acta Automatica Sinica, 2023, 49(8): 1679−1687 doi: 10.16383/j.aas.c190367
Citation: Bi Su-Huan, Jiang Yi-Xiang, Yu Shu-Song, Ding Xiang-Qian, Mu Liang-Liang, Wang Bin. A method for reducing over-dried tobacco at head stage of drying process based on reinforcement learning. Acta Automatica Sinica, 2023, 49(8): 1679−1687 doi: 10.16383/j.aas.c190367

基于强化学习的减少烘丝过程中烟丝 “干头” 量的方法

doi: 10.16383/j.aas.c190367
基金项目: 国家重点研发计划 (2017YFA0700601)资助
详细信息
    作者简介:

    毕素环:中国海洋大学信息科学与工程学院博士. 青岛理工大学信息与控制工程学院讲师. 主要研究方向为机器学习与智能控制. E-mail: bisuhuan2016@163.com

    蒋一翔:浙江中烟工业有限责任公司工程师. 主要研究方向为信息系统应用, 信息安全管理. E-mail: jiangyxlunwen@sina.com

    于树松:中国海洋大学信息科学与工程学院副教授. 主要研究方向为人工智能, 智能控制. 本文通信作者. E-mail: yushusong@ouc.edu.cn

    丁香乾:中国海洋大学信息科学与工程学院教授. 主要研究方向为人工智能, 智能控制. E-mail: dingxq1995@vip.sina.com

    牟亮亮:中国海洋大学信息科学与工程学院博士研究生. 主要研究方向为深度学习与数据挖掘. E-mail: merlin_mu@163.com

    王彬:中国海洋大学继续教育学院讲师. 主要研究方向为机器学习与数据挖掘. E-mail: wangbin@ouc.edu.cn

A Method for Reducing Over-dried Tobacco at Head Stage of Drying Process Based on Reinforcement Learning

Funds: Supported by National Key Research and Development Program of China (2017YFA0700601)
More Information
    Author Bio:

    BI Su-Huan Ph.D. at the College of Information Science and Engineering, Ocean University of China. She is a lecturer at the School of Information and Control Engineering, Qingdao University of Technology. Her research interest covers machine learning and intelligent control

    JIANG Yi-Xiang Engineer at the China Tobacco Zhejiang Industrial CO., LTD.. His research interest covers information system application and information security management

    YU Shu-Song Associate professor at the College of Information Science and Engineering, Ocean University of China. His research interest covers artificial intelligence and intelligent control. Corresponding author of this paper

    DING Xiang-Qian Professor at the College of Information Science and Engineering, Ocean University of China. His research interest covers artificial intelligence and intelligent control

    MU Liang-Liang Ph.D. candidate at the College of Information Science and Engineering, Ocean University of China. His research interest covers deep learning and data mining

    WANG Bin Lecturer at the School of Continuing Education, Ocean University of China. His research interest covers machine learning and data mining

  • 摘要: 针对烘丝开始阶段存在的烘丝温度超调、过干烟丝较多等问题, 提出一种基于强化学习 (Reinforcement learning, RL)的减少烟丝“干头” 量的方法. 该方法利用生产实时数据作为输入特征向量感知烘丝生产过程的状态变化, 以烟丝含水率检测值为依据来评价、优化烘丝温度控制策略, 实现对烘丝机温度设定值的在线修正, 优化烘丝开始阶段的温度控制, 有效改善烟丝过干问题. 与烘丝机的自动控制模式和人工干预模式相比, 烟丝含水率的标准偏差比自动控制时降低了44.7%, 比人工干预时降低了14.3%. 实验结果表明烟丝含水率的稳定性有较大提高, 烟丝“干头” 量明显减少, 验证了所提方法的有效性和可行性.
  • 图  1  人工干预时烟丝含水率标准偏差

    Fig.  1  Standard deviation of moisture content level in tobacco when in manual intervention mode

    图  2  制叶丝工段

    Fig.  2  The stage of cut tobacco processing

    图  3  烘丝过程及温度控制

    Fig.  3  Drying process and temperature control flow

    图  4  烘丝温度优化控制策略

    Fig.  4  Optimal control strategy for drying temperature

    图  5  烘丝温度优化控制流程

    Fig.  5  Optimization of temperature control flow

    图  6  烘丝生产系统状态感知

    Fig.  6  Tobacco drying system state perception

    图  7  模型的输入特征向量

    Fig.  7  Input feature vector of the proposed model

    图  8  3 种常用激活函数及设计的奖励函数

    Fig.  8  Three commonly used activation functions and the designed reward functions

    图  9  3 种模式下烘丝机温度和烟丝含水率曲线

    Fig.  9  The dryer temperature and the moisture content level in tobacco when in three control modes

    表  1  烘丝生产系统状态特征

    Table  1  The state features of tobacco drying system

    特征类别生产状态特征特征数
    原料烟丝KLD烘前水分、KLD烘丝流量、叶丝累计量3
    批次编号年、月、日、生产线编号、班组、生产序号6
    配方参数KLD除水量、含水率目标值、干燥能力、干燥因子4
    过程检测量KLD烘丝段蒸汽流量、SIROX蒸汽流量、SIROX烘丝分汽缸压力、SIROX烘丝分汽缸温度、SIROX排潮风机负压值、SIROX后温度、SIROX阀后蒸汽温度、SIROX阀后蒸汽压力、SIROX阀前蒸汽温度、SIROX阀前蒸汽压力、KLD一次减压后蒸汽压力、KLD烘后水分、KLD烘后温度、KLD排潮温度、Ⅰ区工作蒸汽压力、Ⅱ区工作蒸汽压力、Ⅰ区回水温度、Ⅱ区回水温度、Ⅰ区筒壁温度、Ⅱ区筒壁温度、热风风速、热风温度、排潮负压、风选冷却排潮负压、冷却温度、冷却水分26
    设备参数SIROX蒸汽阀门开度、KLD筒转速、KLDⅠ区蒸汽薄膜阀开度、KLDⅡ区蒸汽薄膜阀开度、Ⅰ区筒壁温度设定值、Ⅱ区筒壁温度设定值、热风蒸汽阀门开度、风门开度、排潮开度、风选冷却排潮开度10
    下载: 导出CSV

    表  2  烟丝含水率标准偏差

    Table  2  Standard deviation of moisture content level in tobacco

    控制模式标准偏差
    开机20 min开机30 min开机40 min
    自动控制0.0970.0820.076
    人工干预0.0560.0530.049
    Actor-Critic优化控制0.0510.0450.042
    下载: 导出CSV
  • [1] Zhu W K, Wang Y, Chen L Y, Wang Z G, Li B, Wang B. Effect of two-stage dehydration on retention of characteristic flavor components of flue-cured tobacco in rotary dryer. Drying Technology, 2016, 34(13): 1621-1629 doi: 10.1080/07373937.2016.1138965
    [2] 李晓理, 王康, 于秀明, 苏伟. 基于CPS框架的微粉生产过程多模型自适应控制. 自动化学报, 2019, 45(7): 1354-1365 doi: 10.16383/j.aas.2018.c180387

    Li Xiao-Li, Wang Kang, Yu Xiu-Ming, Su Wei. CPS-based multiple model adaptive control of GGBS production process. Acta Automatica Sinica, 2019, 45(7): 1354-1365 doi: 10.16383/j.aas.2018.c180387
    [3] 国家烟草专卖局. 卷烟工艺规范. 北京: 中国轻工业出版社, 2016.

    State Tobacco Monopoly Administration. Cigarette Making Process Specification. Beijing: China Light Industry Press, 2016.
    [4] Pakowski Z, Druzdzel A, Drwiega J. Validation of a model of an expanding superheated steam flash dryer for cut tobacco based on processing data. Drying Technology, 2004, 22 (1−2): 45-57 doi: 10.1081/DRT-120028212
    [5] Zhou F, Peng H, Ruan W J, Wang D, Liu M Y, Gu Y F, et al. Cubic-RBF-ARX modeling and model-based optimal setting control in head and tail stages of cut tobacco drying process. Neural Computing & Applications, 2016, 30: 1039-1053.
    [6] 赖旭芝, 李爱萍, 吴敏, 雷琪. 基于多目标遗传算法的炼焦生产过程优化控制. 计算机集成制造系统, 2009, 15 (5): 990-997 doi: 10.13196/j.cims.2009.05.160.laixzh.003

    Lai Xu-Zhi, Li Ai-Ping, Wu Min, Lei Qi. Optimization control based on the multi-objective genetic algorithm for coking plant production process. Computer Integrated Manufacturing Systems, 2009, 15 (5): 990-997 doi: 10.13196/j.cims.2009.05.160.laixzh.003
    [7] 廖龙. 基于模糊控制的烘丝机温度优化控制 [硕士学位论文], 西南科技大学, 2018.

    Liao Long. Drying Machine Temperature Optimization Control Based on Fuzzy Control [Master thesis], Southwest University of Science and Technology, 2018.
    [8] 郑坤明, 张秋菊. 基于弹性动力学模型与遗传算法的Delta机器人模糊PID控制. 计算机集成制造系统, 2016, 22(07): 1707-1716 doi: 10.13196/j.cims.2016.07.010

    Zheng Kun-Ming, Zhang Qiu-Ju. Fuzzy PID control of delta robot based on elastic dynamic model and genetic algorithm. Computer Integrated Manufacturing Systems, 2016, 22 (07): 1707-1716 doi: 10.13196/j.cims.2016.07.010
    [9] 王述彦, 师宇, 冯忠绪. 基于模糊PID控制器的控制方法研究. 机械科学与技术, 2011, 30 (01): 166-172 doi: 10.13433/j.cnki.1003-8728.2011.01.035

    Wang Shu-Yan, Shi Yu, Feng Zhong-Xu. A Method for Controlling a Loading System Based on a Fuzzy PID Controller. Mechanical Science and Technology for Aerospace Engineering, 2011, 30 (01): 166-172 doi: 10.13433/j.cnki.1003-8728.2011.01.035
    [10] Ang K H, Chong G, Li Y. PID control system analysis, design, and technology. IEEE Transactions on Control Systems Technology, 2005, 13 (4): 559-576 doi: 10.1109/TCST.2005.847331
    [11] Skogestad S. Simple analytic rules for model reduction and PID controller tuning. Journal of Process Control, 2003, 13 (4): 291-309 doi: 10.1016/S0959-1524(02)00062-8
    [12] Duma R, Trusca M, Dobra P. Tuning and Implementation of PID Controllers using Rapid Control Prototyping. Control Engineering and Applied Informatics, 2011, 13 (4): 64-73
    [13] Murthy TPK, Manohar B. Microwave drying of mango ginger (Curcuma amada Roxb): prediction of drying kinetics by mathematical modelling and artificial neural network. International Journal of Food Science & Technology, 2012, 47 (6): 1229-1236
    [14] Bravo S, Moreno A H. Prediction model based on neural networks for microwave drying process of amaranth seeds. In: Proceedings of the 3rd International Conference on Compute and Data Analysis. New York, USA: ACM, 2019. 88−93
    [15] Balbay A, Avci E, Sahin O, Coteli R. Modeling of drying process of bittim nuts (pistacia terebinthus) in a fixed bed dryer system by using extreme learning machine. International Journal of Food Engineering, 2012, 8 (4)
    [16] 柴天佑. 自动化科学与技术发展方向. 自动化学报, 2018, 44 (11): 1923-1930 doi: 10.16383/j.aas.2018.c180252

    Chai Tian-You. Development directions of automation science and technology. Acta Automatica Sinica, 2018, 44 (11): 1923-1930 doi: 10.16383/j.aas.2018.c180252
    [17] Dai A N, Zhou X G, Liu X D, Liu J Y, Zhang C. Intelligent control of a grain drying system using a GA-SVM-IMPC controller. Drying Technology, 2018, 36 (12), 1413-1435 doi: 10.1080/07373937.2017.1407938
    [18] Li J S, Xiong Q Y, Wang K, Shi X, Liang S. A recurrent self-evolving fuzzy neural network predictive control for microwave drying process. Drying Technology, 2016, 34 (12), 1434-1444 doi: 10.1080/07373937.2015.1122612
    [19] Wu J, Yang S X, Tian F C. An adaptive neuro-fuzzy approach to bulk tobacco flue-curing control process. Drying Technology, 2017, 35 (4), 465-477 doi: 10.1080/07373937.2016.1183211
    [20] Balbay A, Kaya Y, Sahin O. Drying of black cumin (Nigella sativa) in a microwave assisted drying system and modeling using extreme learning machine. Energy, 2012, 44 (1): 352-357 doi: 10.1016/j.energy.2012.06.022
    [21] Sedighizadeh M, Rezazadeh A. Adaptive PID controller based on reinforcement learning for wind turbine control. In: Proceedings of World Academy of Science Engineering and Technology. Cairo, Egypt: 2008. 257−262
    [22] 陈学松, 杨宜民. 基于执行器-评价器学习的自适应PID控制. 控制理论与应用, 2011, 28 (08): 1187-1192

    Chen Xue-Song, Yang Yi-Min. A novel adaptive PID controller based on Actor-Critic learning. Control and Decision, 2011, 28 (8): 1187-1192
    [23] 孙京诰, 杨嘉雄, 王硕, 薛瑞, 潘红光. 基于Actor-Critic和神经网络的闭环脑机接口控制器设计. 控制与决策, 2018, 33 (11): 1967-1974 doi: 10.13195/j.kzyjc.2017.0791

    Sun Jing-Gao, Yang Jia-Xiong, Wang Shuo, Xue Rui, Pan Hong-Guang. Design of closed-loop brain machine interface controller based on Actor Critic and neural network. Control and Decision, 2018, 33 (11): 1967-1974 doi: 10.13195/j.kzyjc.2017.0791
    [24] 高阳, 陈世福, 陆鑫. 强化学习研究综述. 自动化学报, 2004, 30 (1): 86-100 doi: 10.16383/j.aas.2004.01.011

    Gao Yang, Chen Shi-Fu, Lu Xin. Research on reinforcement learning technology: A review. Acta Automatica Sinica, 2004, 30 (1): 86-100 doi: 10.16383/j.aas.2004.01.011
    [25] Günther J, Pilarski P M, Helfrich G, Hao S, Diepold K. Intelligent laser welding through representation, prediction, and control learning: An architecture with deep neural networks and reinforcement learning. Mechatronics, 2016, 34 : 1-11 doi: 10.1016/j.mechatronics.2015.09.004
    [26] Jiang Y, Fan J L, Chai T Y, Lewis F L. Dual-rate operational optimal control for flotation industrial process with unknown operational model. IEEE Transactions on Industrial Electronics, 2019, 66 (6): 4587-4599 doi: 10.1109/TIE.2018.2856198
    [27] Feng G X, Busoniu L, Guerra T M, Mohammad S. Data-efficient reinforcement learning for energy optimization of power-assisted wheelchairs. IEEE Transactions on Industrial Electronics, 2019, 66 (12): 9734-9744 doi: 10.1109/TIE.2019.2903751
    [28] Zhang K, Zhang H G, Mua Y F, Sun S X. Tracking control optimization scheme for a class of partially unknown fuzzy systems by using integral reinforcement learning architecture. Applied Mathematics and Computation, 2019, 359: 344-356 doi: 10.1016/j.amc.2019.04.084
    [29] 周凯敏, 何晋, 盛科, 余娜, 朱生才, 吉德祥. 滚筒烘丝机内烟丝滞留时间模型的建立及数值模拟. 烟草科技, 2016, 49 (5): 94-99 doi: 10.16135/j.issn1002-0861.20160514

    Zhou Kai-Min, He Jin, Sheng Ke, Yu Na, Zhu Sheng-Cai, Ji De-Xiang. Modeling and numerical simulation of residence time of cut tobacco in cylinder dryer. Tobacco Science & Technology, 2016, 49 (5): 94-99 doi: 10.16135/j.issn1002-0861.20160514
    [30] Barto A G, Sutton R S, Anderson C W. Neuronlike adaptive elements that can solve difficult learning control problems. IEEE Transactions on Systems, Man and Cybernetics, 1983, 13 (5): 834-846
    [31] Konda V R, Tsitsiklis J N. On actor-critic algorithms. SIAM Journal on Control and Optimization, 2003, 42(4): 1143-1166 doi: 10.1137/S0363012901385691
    [32] 刘强, 卓洁, 郎自强, 秦泗钊. 数据驱动的工业过程运行监控与自优化研究展望. 自动化学报, 2018, 44(11): 1944-1956 doi: 10.16383/j.aas.2018.c180207

    Liu Qiang, Zhuo Jie, Lang Zi-Qiang, Qin Si-Zhao. Perspectives on data-driven operation monitoring and self-optimization of industrial processes. Acta Automatica Sinica, 2018, 44(11): 1944-1956 doi: 10.16383/j.aas.2018.c180207
    [33] 陈龙, 刘全利, 王霖青, 赵珺, 王伟. 基于数据的流程工业生产过程指标预测方法综述. 自动化学报, 2017, 43(6): 944-954 doi: 10.16383/j.aas.2017.c170136

    Chen Long, Liu Quan-Li, Wang Lin-Qing, Zhao Jun, Wang Wei. Data-driven prediction on performance indicators in process industry: a survey. Acta Automatica Sinica, 2017, 43(6): 944-954 doi: 10.16383/j.aas.2017.c170136
  • 加载中
图(9) / 表(2)
计量
  • 文章访问数:  287
  • HTML全文浏览量:  67
  • PDF下载量:  121
  • 被引次数: 0
出版历程
  • 收稿日期:  2019-05-14
  • 录用日期:  2019-09-02
  • 网络出版日期:  2023-07-28
  • 刊出日期:  2023-08-21

目录

    /

    返回文章
    返回