摘要
As numerous distributed energy resources(DERs)are integrated into the distribution networks,the optimal dispatch of DERs is more and more imperative to achieve transition to active distribution networks(ADNs).Since accurate models are usually unavailable in ADNs,an increasing number of reinforcement learning(RL)based methods have been proposed for the optimal dispatch problem.However,these RL based methods are typically formulated without safety guarantees,which hinders their application in real world.In this paper,we propose an RL based method called supervisor-projector-enhanced safe soft actor-critic(S3AC)for the optimal dispatch of DERs in ADNs,which not only minimizes the operational cost but also satisfies safety constraints during online execution.In the proposed S3AC,the data-driven supervisor and projector are pre-trained based on the historical data from supervisory control and data acquisition(SCADA)system,effectively providing enhanced safety for executed actions.Numerical studies on several IEEE test systems demonstrate the effectiveness and safety of the proposed S3AC.
基金
supported in part by the National Key Research and Development Plan of China(No.2022YFB2402900)
in part by the Science and Technology Project of State Grid Corporation of China“Key Techniques of Adaptive Grid Integration and Active Synchronization for Extremely High Penetration Distributed Photovoltaic Power Generation”(No.52060023001T)。
作者简介
Xu Yang received the B.S.degree from the Electrical Engineering Depart ment,Tsinghua University,Beijing,China,in 2022,where he is currently pursuing the Ph.D.degree.His research interests include active distribution system operation and control,safe and robust reinforcement learning and its application in the energy system.e-mail:yangxuthu@163.com;Haotian Liu received the B.S.and Ph.D.degrees from the Electrical Engi neering Department,Tsinghua University,Beijing,China,in 2018 and 2023,respectively.He is currently a Postdoctor with Tsinghua University.His re search interests include active distribution system operation and control,model-free control,machine learning especially reinforcement learning,and their applications in the energy system.e-mail:liuhaotian@tsinghua.edu.cn;corresponding author:Wenchuan Wu received the B.S.,M.S.,and Ph.D.degrees from the Electrical Engineering Department,Tsinghua University,Beijing,China,in 1996,1998,and 2023,respectively.He is currently a Full Professor with Tsinghua University.His research interests include energy management system,active distribution system operation and control,machine learning and its application in energy system.e-mail:wuwench@tsinghua.edu.cn;Qi Wang received the B.S.degree from the School of Electrical Engineer ing and Automation,Harbin Institute of Technology,Harbin,China,in 2019.He is currently pursuing the Ph.D.degree in the Electrical Engineer ing Department,Tsinghua University,Beijing,China.His research interests include optimization and control in hierarchical power system with integra tion of renewable generation.e-mail:wangq19@mails.tsinghua.edu.cn;Peng Yu received the Ph.D.degree in electrical and electronic engineering from Dalian University of Technology,Dalian,China,in 2012.He is cur rently with the State Grid Shandong Electric Power Company,Jinan,China.His research interests include distributed generation,microgrid,and integrat ed energy system.e-mail:167274738@qq.com;Jiawei Xing received the M.S.degree from the China-EU Institute for Clean and Renewable Energy,Huazhong University of Science and Technol ogy(HUST),Wuhan,China,in 2019.He is currently with the State Grid Shandong Electric Power Company,Jinan,China.His research interests in clude distributed generation,microgrid,and integrated energy system.e-mail:573602466@qq.com;Yuejiao Wang received the M.S.degree in electrical engineering from North China Electric Power University,Beijing,China,in 2015.Currently,she works at the Electric Power Science Research Institute of State Grid Shandong Electric Power Company,Jinan,China.Her research interests in clude distributed photovoltaic scheduling and operation management.e-mail:wangyuejiao2012@126.com。
