Distributed Neural Networks Training for Robotic Manipulation With Consensus Algorithm

【Author】 Liu, Wenxing; Niu, Hanlin; Jang, Inmo; Herrmann, Guido; Carrasco, Joaquin

【Source】IEEE TRANSACTIONS ON NEURAL NETWORKS AND LEARNING SYSTEMS

【影响因子】14.255

【Abstract】In this article, we propose an algorithm that combines actor-critic-based off-policy method with consensus-based distributed training to deal with multiagent deep reinforcement learning problems. Specifically, convergence analysis of a consensus algorithm for a type of nonlinear system with a Lyapunov method is developed, and we use this result to analyze the convergence properties of the actor training parameters and the critic training parameters in our algorithm. Through the convergence analysis, it can be verified that all agents will converge to the same optimal model as the training time goes to infinity. To validate the implementation of our algorithm, a multiagent training framework is proposed to train each Universal Robot 5 (UR5) robot arm to reach the random target position. Finally, experiments are provided to demonstrate the effectiveness and feasibility of the proposed algorithm.

【Keywords】Training; Reinforcement learning; Convergence; Task analysis; Robot kinematics; Manipulators; Privacy; Consensus; deep reinforcement learning; Lyapunov methods; manipulator; multiagent systems

【发表时间】

【收录时间】2022-08-15

【文献类型】理论模型

【主题类别】

区块链技术-协同技术-机器学习