AUTHOR=Zhang Zhiyuan , Wu Yongjun , Hao Zhenkun , Song Minghui , Yu Peipei 

TITLE=Safe dynamic optimization of automatic generation control via imitation-based reinforcement learning

JOURNAL=Frontiers in Energy Research

VOLUME=Volume 12 - 2024

YEAR=2024

URL=https://www.frontiersin.org/journals/energy-research/articles/10.3389/fenrg.2024.1464151

DOI=10.3389/fenrg.2024.1464151

ISSN=2296-598X

ABSTRACT=The increasing penetration of distributed generations (e.g., solar power and wind power) causes unpredictable disturbances in power systems, which accelerates the application of intelligent control in automatic generation control (AGC), such as reinforcement learning (RL). However, traditional RL cannot ensure constraint safety during training and frequently violates constraint (e.g., frequency limitations), further threatening the grid operation's safety and stability. To address the safety issue, this paper proposes a novel safe RL framework to combine expert experiences with the RL controller, forming imitation-based RL. This method obtains an initialized safe policy by imitating the expert experience, which prevents random explorations at the beginning. Specifically, we first formulate the AGC problem as a Markov decision process in a mathematical way. Then, the imitation mechanism is developed on top of soft actor-critic RL algorithm. Finally, numerical studies on an IEEE 39-bus network show that, the proposed method can better satisfy the frequency control performance standard, with improving the RL training efficiency.