Hierarchical Reinforcement Learning for Blockchain-Assisted Software Defined Industrial Energy Market

【Author】 Cao, Yifan; Ren, Xiaoxu; Qiu, Chao; Wang, Xiaofei

【Source】IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS

【影响因子】11.648

【Abstract】Energy Internet (EI) is developing and booming rapidly with the increase of distributed energy resources, which is beneficial to address the severe condition of industrial energy. However, there are inevitable credit crises and utility optimization challenges in EI that need to be settled. In this article, we propose a blockchain-assisted software defined energy Internet (BSDEI), where a distributed energy market smart contract is designed to ensure transactions executed reliably and participants' accounts dealt accurately. In order to jointly optimize the utilities of operators, retailers, and industrial prosumers in BSDEI, we formulate the whole trading process as a three-stage Stackelberg game, with the proof of existence and uniqueness for the Stackelberg equilibrium. Then, we design a hierarchical distributed policy gradient algorithm to solve the Stackelberg game under incomplete information. We implement a blockchain-based industrial energy trading system using a middleware platform. The smart contract is deployed on the consortium blockchain, providing website interfaces for participants to operate. Furthermore, we conduct experiments for analyzing economic benefits. Our system prototype demonstrates the feasibility of BSDEI and the algorithm exceeds about 18% in total mean reward than comparing algorithms.

【Keywords】Games; Blockchains; Smart contracts; Software; Costs; Optimization; Reinforcement learning; Blockchain; industrial energy market; reinforcement learning; software defined network (SDN); Stackelberg game

【发表时间】2022 SEP

【收录时间】2022-06-26

【文献类型】实证性文章

【主题类别】

区块链应用-实体经济-能源领域