Home > View > Focus

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

2023-01-12

【论文推荐】华北电力大学董雷等：基于多智能体深度强化学习的电-气-热综合能源系统分布式优化

原创英文期刊编辑部全球能源互联网期刊 2023-01-12 08:00

摘要

电-气-热综合能源系统协调优化问题具有强耦合、非凸非线性等特点，集中式优化方法通讯成本高，建模复杂，传统的数值迭代求解在应对不确定性、求解效率上存在一定局限，难以在线应用。本文针对电-气-热综合能源系统协调优化问题，构建了一种含动态分配因子的综合能源分布式系统模型，采用基于多智能体深度确定性策略梯度方法，将优化问题设计为多智能体强化学习环境下的分布式优化问题。引入动态分配因子可以使得系统能够考虑到实时供需情况变化对系统优化的影响，动态协调不同能源进行互补利用，有效提升系统的经济性。采用多决策中心的分布式模型能够在减轻系统通讯压力的同时达到和集中式优化近乎一致的结果。所提优化方法能够在训练中考虑到可再生能源和负荷的双重不确定性，训练后的模型相比于传统迭代求解方法不仅具有较好应对不确定性的能力，还能够实现系统实时决策，有利于在线应用。最后，通过耦合三个能量枢纽智能体的综合能源系统进行算例分析，验证了所提方法的有效性。

Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning

基于多智能体深度强化学习的电-气-热综合能源系统分布式优化

Lei Dong 1, Jing Wei 1, Hao Lin1, Xinying Wang2

(1.School of Electric Engineering, North China Electric Power University, Changping District, Beijing 102206, P. R. China

2.School of Electric Engineering, North China Electric Power University, Changping District, Beijing 102206, P. R. China)

收听作者1分钟语音介绍

Distributed optimization of electricity-Gas-Heat integrated

Abstract

The coordinated optimization problem of the electricity-gas-heat integrated energy system (IES) has the characteristics of strong coupling, non-convexity, and nonlinearity. The centralized optimization method has a high cost of communication and complex modeling. Meanwhile, the traditional numerical iterative solution cannot deal with uncertainty and solution efficiency, which is difficult to apply online. For the coordinated optimization problem of the electricity-gas- heat IES in this study, we constructed a model for the distributed IES with a dynamic distribution factor and transformed the centralized optimization problem into a distributed optimization problem in the multi-agent reinforcement learning environment using multi-agent deep deterministic policy gradient. Introducing the dynamic distribution factor allows the system to consider the impact of changes in real-time supply and demand on system optimization, dynamically coordinating different energy sources for complementary utilization and effectively improving the system economy. Compared with centralized optimization, the distributed model with multiple decision centers can achieve similar results while easing the pressure on system communication. The proposed method considers the dual uncertainty of renewable energy and load in the training. Compared with the traditional iterative solution method, it can better cope with uncertainty and realize real- time decision making of the system, which is conducive to the online application. Finally, we verify the effectiveness of the proposed method using an example of an IES coupled with three energy hub agents.

Keywords

Integrated energy system, Multi-agent system, Distributed optimization, Multi-agent deep deterministic policy gradient, Real-time optimization decision.

Fig.1 Schematic diagram of IES distributed structure based on multi-agents

Fig.2 Schematic diagram of the compressor model

Fig.3 MADDPG algorithm diagram

Fig.4 Solution flow chart of IES distributed optimization model based on MADDPG

Fig.5 A case of distributed optimization system for IES with three EH agents

Fig.6 Outputs of wind power and PV

Fig.7 The overall reward value of the agent

Fig.8 The overall penalty of the agent

Fig.9 Optimization results of agent 1 and agent 2

Fig.10 Optimization results of agent 1 in the extreme scenario

Fig.11 Comparison of training results between DDPG and MADDPG

本文引文信息

Dong L, Wei J, Lin H, et al. (2022) Distributed optimization of electricity-Gas-Heat integrated energy system with multi-agent deep reinforcement learning, 5(6): 604-617

董雷，魏静，林灏等 (2022) 基于多智能体深度强化学习的电-气-热综合能源系统分布式优化. 全球能源互联网（英文）, 5(6): 604-617

Biographies

Lei Dong

Lei Dong received her master degree at Tianjin University, Tianjin, China. She is currently an associate Professor in the Electrical Engineering Department with North China Electric Power University, Beijing, China. Her research interests include power systems analysis and control, power system optimal dispatch and operation control, application of artificial intelligence in power system.

Jing Wei

Jing Wei received master degree in the School of Electrical and Electronic Engineering, North China Electric Power University, Beijing, China, in 2022. Her research interests include application of artificial intelligence in power system.

Hao Lin

Hao Lin is currently working towards the master degree in the School of Electrical and Electronic Engineering, North China Electric Power University, Beijing, China. His research interests include power system optimal dispatch and application of artificial intelligence in power system.

Xinying Wang

Xinying Wang received the PhD degree at Dalian University of Technology in 2015, Dalian, China. He is working in Artificial Intelligence Application Research Department of China Electric Power Research Institute (CEPRI). His research interests include artificial intelligence and its application in energy internet.

编辑：王彦博

审核：王伟

郑重声明

根据国家版权局相关规定，纸媒、网站、微博、微信公众号转载、摘编本网站作品，需包含本网站名称、二维码等关键信息，并在文首注明《全球能源互联网》原创。个人请按本网站原文转发、分享。