MARL Sim2real Transfer: Merging Physical Reality With Digital Virtuality in Metaverse

【Author】 Shi, Haoran; Liu, Guanjun; Zhang, Kaiwen; Zhou, Ziyuan; Wang, Jiacun

【Source】IEEE TRANSACTIONS ON SYSTEMS MAN CYBERNETICS-SYSTEMS

【影响因子】11.471

【Abstract】Metaverse is an artificial virtual world mapped from and interacting with the real world. In metaverse, digital entities coexist with their physical counterparts. Powered by deep learning, metaverse is inevitably becoming more intelligent in the interactions between reality and virtuality. However, it is confronted with a nontrivial problem known as sim2real transfer when deep learning techniques try to bridge the reality gap between the physical world and simulations. In this article, we use multiagent deep reinforcement learning (MARL) to implement collective intelligence for digital entities as well as their physical counterparts. To model the immersive environments in metaverse, we define a nonstationary variant of Markov games and propose a recurrent MARL solution to it. Based on the solution, MARL sim2real transfer that bridges real and virtual multiple unmanned aerial vehicle (multi-UAV) systems is successfully conducted by employing recurrent multiagent deep deterministic policy gradient (R-MADDPG) with the domain randomization technique. Additionally, we use perception-control modularization to improve the generalization performance of MARL policies and make training more efficient.

【Keywords】Metaverse; Games; Markov processes; Task analysis; Robots; Vehicle dynamics; Training; multiagent deep reinforcement learning (MARL); multiple unmanned aerial vehicle (multi-UAV); nonstationary Markov game; sim2real transfer

【发表时间】

【收录时间】2023-01-17

【文献类型】实验仿真

【主题类别】

区块链应用-虚拟经济-元宇宙