AUTHOR=Ma Ming , Du Wanlin , Wang Ling , Ding Cangbi , Liu Siqi 

TITLE=Research on the multi-timescale optimal voltage control method for distribution network based on a DQN-DDPG algorithm

JOURNAL=Frontiers in Energy Research

VOLUME=Volume 10 - 2022

YEAR=2023

URL=https://www.frontiersin.org/journals/energy-research/articles/10.3389/fenrg.2022.1097319

DOI=10.3389/fenrg.2022.1097319

ISSN=2296-598X

ABSTRACT=Since a large number of distributed generators (GDs) such as distributed Photovoltaic (DPV) and energy storage (ES) are connected to the distribution networks (DNs), the voltage overlimit problem of DN with high permeability GDs is more prominent. Combined with the rapid development of deep reinforcement learning, this paper proposes a multi-time scale voltage optimal control method of distribution network based on DQN-DDPG algorithm. Firstly, the output characteristics of the devices with voltage regulation function in the DN are analyzed, and the DQN algorithm to optimize the voltage regulation mode for a long time and Deep Deterministic Policy Gradient (DDPG) algorithm to optimize the voltage regulation mode for a short time are described. Secondly, the design of DQN-DDPG algorithm based on Markov decision process transformation based on objectives and constraints is presented under the premise of considering ES state of charge (SOC) to prolong the life of ES. Finally, the proposed strategy is verified on the simulation platform, and the results obtained by using particle swarm optimization (PSO) algorithm are compared, which show the effectiveness and advancement of the proposed strategy.