Coordinated AGC Algorithm for Distributed Multi-region Multi-energy Micro-network Group
-
摘要: 综合能源多区域协同是电网发展趋势, 而核心问题是采用何种方法对多区域进行协同. 本文基于Q (
$\sigma $ )融入了资格迹及双重Q学习, 提出一种面向多区域多能微网群的多智能体协同控制算法, 即DQ ($\sigma ,\lambda $ ), 避免传统强化学习动作探索值高估的同时, 来获取分布式多区域的协同. 通过对改进的IEEE两区域负荷频率控制模型及三区域多能微网群自动发电控制(Automatic generation control, AGC)模型仿真, 结果表明, 与传统方法相比, 所提算法具有快速收敛性和更优动态性能, 能获得分布式多区域多能微网群的协同.Abstract: Comprehensive energy multi-region coordination is the development trend of the power grid, and the core question is what method to use for multi-region coordination. Based on the integration of the qualification trace and dual Q-learning in Q ($\sigma $ ), this paper proposes a multi-agent collaborative control algorithm for multi-region and multi-energy micro-grid group, named DQ ($\sigma ,\lambda$ ), to avoid high exploration value of traditional reinforcement learning actions. At the same time of evaluation, the distributed multi-region collaboration is obtained. Simulations of the improved IEEE two area load frequency control model and the three area multi-energy microgrid group automatic generation control (AGC) model show that the proposed algorithm has fast convergence and better dynamic performance than traditional methods, and can achieve distributed Synergy of regional multi-energy microgrid groups. -
图 1 多能微网群多区域协同控制架构
Fig. 1 Multi-energy microgrid group multi-region cooperative control architecture
图 4 改进的IEEE标准两区域负荷频率控制模型
Fig. 4 Improved IEEE standard two-area load frequency control model
图 6 阶跃负荷扰动下不同算法的性能指标
Fig. 6 Performance index of different algorithms under step load disturbance
图 7 随机白噪声扰动下不同算法的控制性能
Fig. 7 Control performance of different algorithms under stochastic white noise disturbance
图 8 分布式3区域多能微网群协同AGC模型
Fig. 8 Coordinated AGC model of a distributed three-area multi-energy microgrid group
表 1 模型传递函数的参数
Table 1 Parameters of the model transfer function
机组 参数 数值 小水电机组 二次时延TSH 3 伺机电动机时间常数TP 0.04 伺机增益KS 5 永态转差系数RP 1 复位时间TR 0.3 暂态转差系数RT 1 闸门最大开启率Rmaxopen/(pu/s) 0.16 闸门最大关闭率Rmaxclose/(pu/s) 0.16 机组启动时间TWH 1 生物发电机组 二次时延TSB 10 调速器的时间常数TGB 0.08 蒸汽启动时间TWB 5 机械启动时间TMB 0.3 微型燃气轮机机组 二次时延TSM 5 燃油系统滞后时间常数T1 0.8 燃油系统滞后时间常数T2 0.3 负荷限制时间常数T3 3 温度控制环路增益KT 1 负荷限制Lmax 1.2 燃料电池机组 二次时延TSF 2 调速器的时间常数TF 10.056 逆变器增益KF 9.205 柴油发电储能机组 二次时延TSD 7 调速器的时间常数TGD 2 蒸汽启动时间TWF 1 机械启动时间TMD 3 
下载: 导出CSV
表 2 AGC机组参数
Table 2 AGC unit parameters
区域 类型 机组序号 $\Delta P_{\rm{in}}^{\max }$
(kW/s)$\Delta P_{\rm{in}}^{\min }$
(kW/s)$\Delta P_{\rm{in}}^{\rm{rate }+ }$
(kW/s)$\Delta P_{\rm{in}}^{\rm{rate} - }$
(kW/s)区域1和区域3 小水电 G1 250 − 250 15 − 15 G2 250 − 250 15 − 15 G3 150 − 150 8 − 8 G4 150 − 150 8 − 8 G5 150 − 150 8 − 8 G6 100 − 100 7 − 7 G7 100 − 100 7 − 7 微型燃气轮机 G8 100 − 100 1.2 − 1.2 G9 100 − 100 1.2 − 1.2 G10 150 − 150 1.8 − 1.8 G11 150 − 150 1.8 − 1.8 燃料电池 G12 200 − 200 7 − 7 G13 200 − 200 7 − 7 G14 150 − 150 6 − 6 G15 150 − 150 6 − 6 区域2 小水电 G1 250 − 250 15 − 15 G2 250 − 250 15 − 15 G3 150 − 150 8 − 8 G4 150 − 150 8 − 8 G5 150 − 150 8 − 8 G6 100 − 100 7 − 7 柴油发电机储 G7 250 − 250 2 − 2 G8 250 − 250 2 − 2 G9 120 − 120 1 − 1 G10 120 − 120 1 − 1 生物质能 G11 200 − 200 3 − 3 G12 200 − 200 3 − 3 G13 200 − 200 3 − 3 G14 200 − 200 3 − 3
下载: 导出CSV
-
[1] Meng L X, Savaghebi M, Andrad F, Vasquez J C, Guerrero J M, Graells M. Microgrid central controller development and hierarchical control implementation in the intelligent microgrid lab of Aalborg University. In: Proceedings of the 2015 IEEE Applied Power Electronics Conference and Exposition (APEC), Charlotte, NC, USA: IEEE, 2015. 2585−2592 [2] Brijesh P, Jiju K, Dhanesh P R, Joseph A. Microgrid for sustainable development of remote villages. In: Proceedings of the 2019 IEEE Region 10 Conference, Kochi, India: IEEE, 2019. 2433−2438 [3] Wang J, Cisse B M, Brown D, Crabb A. Development of a microgrid control system for a solar-plus-battery microgrid to support a critical facility. In: Proceedings of the 2017 IEEE Power and Energy Society Innovative Smart Grid Technologies Conference (ISGT), Washington, DC, USA: IEEE, 2017. 1−5 [4] Suyanto H, Irawati R. Study trends and challenges of the development of microgrids. In: Proceedings of the 6th IEEE International Conference on Advanced Logistics and Transport (ICALT), Bali, Indonesia: IEEE, 2017. 160−164 [5] Behera A, Panigrahi T K, Ray P K, Sahoo A K. A novel cascaded PID controller for automatic generation control analysis with renewable sources. IEEE/CAA Journal of Automatica Sinica, 2019, 6(6): 1438−1451 doi: 10.1109/JAS.2019.1911666 [6] Jagatheesan K, Anand B, Samanta S, Dey N, Ashour A S, Balas V E. Design of a proportional-integral-derivative controller for an automatic generation control of multi-area power thermal systems using firefly algorithm. IEEE/CAA Journal of Automatica Sinica, 2019, 6(2): 503−515 doi: 10.1109/JAS.2017.7510436 [7] 赵熙临, 林震宇, 付波, 何莉, 徐光辉. 预测优化PID方法在含风电电力系统AGC中的应用. 电力系统及其自动化学报, 2019, 31: 16−22Zhao Xi- Lin, Lin Zhen-Yu, Fu Bo, He Li, Xu Guang-Hui. Application of predictive optimization PID method to AGC of power system with windy power. Journal of Power System and Automation, 2019, 31: 16−22 [8] 谢平平, 李银红, 刘晓娟, 石东源, 段献忠. 基于社会学习自适应细菌觅食算法的互联电网AGC最优PI/PID控制器设计. 中国电机工程学报, 2016, 36(20): 5440−5448Xie Ping-Ping, Li Yin-Hong, Liu Xiao-Juan, Shi Dong-Yuan, Duan Xian-Zhong. Optimal PI/PID controller design of AGC based on social learning adaptive bacteria foraging algorithm for interconnected power grids. Proceedings of the Chinese Society of Electrical Engineering, 2016, 36(20): 5440−5448 [9] Arya Y. A novel CFFOPI-FOPID controller for AGC performance enhancement of single and multi-area electric power systems. ISA Transactions, 2020, 100: 126−135 [10] Xi L, Yu L, Xu Y C, Wang S X, Chen X. A novel multi-agent DDQN-AD method-based distributed strategy for automatic generation control of integrated energy systems. IEEE Transactions on Sustainable Energy, 2019, DOI: 10.1109/TSTE.2019.2958361 [11] 吴新, 史军, 马伟哲, 陈俊斌. 基于极限Q学习算法的微电网自动发电控制. 新型工业化, 2019, 9(4): 22−26Wu Xin, Shi Jun, Ma Wei-Zhe, Chen Jun-Bin. Automatic generation control of micro grid based on extreme Q-learning algorithm. The Journal of New Industrialization, 2019, 9(4): 22−26 [12] 余涛, 梁海华, 周斌. 基于R(λ) 学习的孤岛微电网智能发电控制. 电力系统保护与控制, 2012, 40(13): 7−13 doi: 10.7667/j.issn.1674−3415.2012.13.002Yu Tao, Liang Hai-Hua, Zhou Bin. Smart power generation control for microgrids islanded operation based on R(λ) learning. Power System Protection and Control, 2012, 40(13): 7−13 doi: 10.7667/j.issn.1674−3415.2012.13.002 [13] 吴丽珍, 雷艾虎, 郝晓弘. 基于模型预测控制的孤岛微电网频率二次控制策略. 兰州理工大学学报, 2019, 45(6): 99−107 doi: 10.3969/j.issn.1673−5196.2019.06.018Wu Li-Zhen, Lei Ai-Hu, Hao Xiao-Hong. Secondary control strategy of microgrid frequency of isolated island based on model predictive control. Journal of Lanzhou University of Technology, 2019, 45(6): 99−107 doi: 10.3969/j.issn.1673−5196.2019.06.018 [14] 李文浩. 去中心化多智能体强化学习算法研究[硕士学位论文]. 华东师范大学, 中国, 2019.Li Wen-hao. Decentralized Multi-Agent Reinforcement Learning Algorithm Research. [Master thesis]. East China Normal University, China, 2019. [15] 綦晓. 基于多智能体系统及自抗扰控制理论的微网负荷频率控制策略研究[博士学位论文]. 华北电力大学(北京), 中国, 2019.Qi Xiao. Research on Microgrid Load Frequency Control Strategy Based on Multi-Agent System and Active Disturbance Rejection Control Algorithm [Ph.D. dissertation]. North China Electric Power University, China, 2019. [16] 曹倩. 多智能体系统一致性算法及其在微网中的应用[博士学位论文]. 电子科技大学, 中国, 2016.Cao Qian. Consensus Algorithms Of Multi-Agent Systems And Its Application On Micro-Grid [Ph.D. dissertation]. University of Electronic Science and Technology of China, China, 2016. [17] 衣楠. 微网分布式协调控制系统设计及仿真实现[硕士学位论文]. 华北电力大学, 中国, 2014.Yi Nan. Design and Simulation of Microgrid Distributed Coordination Control System [Master thesis]. North China Electric Power University, China, 2014. [18] 李楠芳. 基于多智能体技术的微电网控制算法的研究[硕士学位论文]. 华北电力大学, 中国, 2011.Li Nan-Fang. Research on Control Algorithms Based on Multi-agent Technology of Microgrid [Master thesis]. North China Electric Power University, China, 2011. [19] Xi L, Li Y D, Huang Y H, Lu L, Chen J F. A novel automatic generation control method based on the ecological population cooperative control for the islanded smart grid. Complexity, 2018, 2018: 1−17 [20] Watkins C J C H. Learning from Delayed Rewards. [Ph.D. dissertation]. King's College, Cambridge, England, 1989. [21] De Asis K, Hernandez-Garcia J F, Holland G Z, Sutton R S. Multi-step reinforcement learning: A unifying algorithm. AAAI, 2018, arXiv: 1703.01327 [22] Hasselt H V. Double Q-learning. Neural Information Processing Systems 23, Curran Associates, Inc. 2613–2621 [23] Sutton R S. Learning to predict by the methods of temporal differences. Machine Learning, 1988, 3(1): 9–44 [24] Van Seijen H, Van Hasselt H, Whiteson S, Wiering M A. A theoretical and empirical analysis of expected sarsa. In: Proceedings of the 2009 IEEE Symposium Conference on Adaptive Dynamic Programming and Reinforcement Learning. 2009. 177−184 [25] Jaleeli N, Vanslyck L S. NERC's new control performance standards. IEEE Transactions on Power Systems, 1999, 14(3): 1091−1099 [26] Zhang X S, Yu T, Pan Z N, Yang B, Bao T. Lifelong learning for complementary generation control of interconnected power grids with high-penetration renewables and EVs. IEEE Transactions on Power Systems, 2018, 33(4): 4097−4110 doi: 10.1109/TPWRS.2017.2767318 [27] 黄际元. 储能电池参与电网调频的优化配置及控制策略研究[博士学位论文]. 湖南大学, 中国, 2015.Huang Ji-Yuan. Study on Optimal Allocation and Control Strategy Design of Battery Energy Storage System for Power Grid Frequency Regulation [Ph.D. dissertation]. Hunan University, China, 2015. [28] Sun Q Y, Huang B N, Li D S, Ma D H, Zhang Y B. Optimal placement of energy storage devices in microgrids via structure preserving energy function, IEEE Transactions on Industrial Informatics, 2016, 12(3): 1166−1179 [29] Xu D, Wu Q, Zhou B, Li C, Bai L, Huang S. Distributed multi-energy operation of coupled electricity, heating and natural gas networks, IEEE Transactions on Sustainable Energy, 2019, DOI: 10.1109/TSTE.2019.2961432 [30] Yu T, Zhou B, Chan K W, Chen L, Yang B. Stochastic optimal relaxed automatic generation control in non-Markov environment based on multi-step Q (λ) learning. IEEE Transactions on Power Systems, 2011, 26 (3): 1272−1282 [31] Sun Q Y, Han R K, Zhang H G, Zhou J G, Guerrero J M. A multi-agent-based consensus algorithm for distributed coordinated control of distributed generators in the energy internet. IEEE Transactions on Smart Grid, 2015, 6(6): 3006−3019 doi: 10.1109/TSG.2015.2412779 [32] Saha A K, Chowdhury S, Chowdhury S P, Crossley A. Modelling and simulation of microturbine in islanded and grid-connected mode as distributed energy resource. In: Proceedings of the 2008 IEEE Power and Energy Society General Meeting-Conversion and Delivery of Electrical Energy in the 21st Century. Pittsburgh, PA, USA: IEEE, 2008. 1−7 [33] Zhang X S, Li Q, Yu T, Yang B. Consensus transfer Q-learning for decentralized generation command dispatch based on virtual generation tribe. IEEE Transactions on Smart Grid, 2018, 9(3): 2152−2165 -
点击查看大图
图(11) / 表(2)
计量
- 文章访问数: 1694
- HTML全文浏览量: 769
- PDF下载量: 277
- 被引次数: 0
